Heat treatment monitoring system

ABSTRACT

A heat treatment monitoring system comprises a heat treatment machine comprising a heat treatment chamber, a double glass window comprising an inside window and an outside window, and an illumination apparatus for illuminating the inside of the heat treatment chamber, and a monitoring apparatus mounted to the heat treatment machine and comprising a camera to observe the inside of the heat treatment chamber through the inside window, wherein the visible transmittance of the outside window is lower than the visible transmittance of the inside window to reduce reflections within the double glass window structure and outside illumination effects on image processing of images recorded by the camera.

The present invention is related to a heat treatment monitoring system,in particular a monitoring system for heating, baking or proofing offood to be heated like bread, dough or the like.

Treating food with heat is done by mankind probably since the inventionof fire. However, up until now this task is still controlled by a humanoperator. The goal of the underlying invention is to automate the foodtreatment and in particular bread baking or proofing such that no humaninteraction is necessary.

Many inventions are known that get close to this goal. For instance, DE10 2005 030483, describes an oven for heat treatment with an openingdevice that can be opened or closed automatically. In DE 20 2011 002570,an apparatus for heat treatment of food products and receiving them on aproduct carrier is stated. The latter is equipped with a control systemfor controlling a treatment process for detecting the type and amount ofproducts. The controller selects with predefined data to perform anautomatic identification of the products. A camera outside of thetreatment chamber may be used as sensor. In EP 250 169 A1, a baking ovendoor is described that incorporates a camera to visualize a muffle orbaking chamber. The visualization is of advantage to save energy lossescreated by looking windows. US 0 2011 0123 689 describes an oven thatcomprises a camera and a distance sensor in order to extract productfeatures for heating processes. DE 20 2011 002 570 U1 describes a systemwith sensor acquisition in ovens.

However, still the heat treatment of food in particular for baking breadwith an oven follows manual setup and happens under human supervision.When a human operator puts bread into an oven, important properties suchas temperature, time, and circulation have to be set. Usually thesettings are stored within a database of oven control programs. A humanoperator has to pick the appropriate program and this still is source oferror and creates human labor with a certain degree of know how.Further, many process parameters may lead to an undesired food productoutcome. Bread may be under baked or over baked, even if the correctprogram has been chosen. This may be caused by differences in ovenpre-heating, dough preparation, outside temperature, outside humidity,load distribution, oven door opening times and many more. It stillrequires skilled human labor to supervise baking or food heat treatment.

Moreover, when processing food as e.g. in a manufacturing plant for rawor prebaked dough, the objects being processed underlie many processvariations. Due to the nature of many food products, the objects beingprocessed may vary in shape, colour, size and many other parameters.This is one of the key challenges in industrial food processing, becauseoften processing devices have to be adjusted to compensate thesevariations. Hence, it is desirable to automate the industrial processingsteps, making manual adjustments ideally unnecessary. In baking, forinstance changes in flour characteristics may result in severe processvariations of industrial dough processing devices. For instance it maybe necessary to adapt parameters of a mixer, a dough divider, doughforming devices, proofing, cutter, packaging, the baking program of anoven or a vacuum baking unit.

In order to achieve the goal of automated baking or food processing itis necessary to provide the corresponding monitoring system with datafrom suitable monitoring devices. Hence, there is a need for monitoringsystems with monitoring devices for collecting suitable data.

For goods baked in an oven a monitoring system with a camera may be usedto monitor the baking process through a window in an oven. However, inorder to prevent thermal losses by heat dissipation through the window,in conventional ovens such looking windows are made of double glass,i.e. they have an inner and an outer glass pane. Hence, light fromoutside the oven may pass the outer glass pane and be reflected into thecamera by the inner glass pane, leading to disturbed images of the bakedgoods.

It is therefore desirable to provide a heat treatment monitoring systemthat reduces disturbances of images of the baked goods captured througha double glass window.

In food processing systems data concerning the structure of theprocessed food should be obtained without stopping the food processing,in order to not reduce a production output. It is hence desirable toadjust the parameters of the aforementioned devices of a food processingsystem or any other device in food processing, based on contactlessmeasurement techniques.

In order to make data captured by monitoring devices useful forautomated baking or food processing it is desirable to provide a methodfor classifying a multitude of images recorded by monitoring devicesobserving a processing area of processed food and to provide a machineusing the same.

Once the data are suitably classified it is desirable to take advantageof cognitive capabilities in order to increase the heat treatmentmachine in flexibility, quality, and efficiency. This can be furtherseparated in the objects:

It is desirable to provide a system being able to gain knowledge bylearning from a human expert how to abstract relevant information withinfood processing and how to operate an oven, wherein the system shouldshow reasonable behavior in unknown situations and should be able tolearn unsupervised.

It is desirable to provide a system increasing the efficiency byclosed-loop control of energy supply adapting to changes in processingtime and maintaining a desired food processing state.

It is desirable to provide a system having flexibility for individuallydifferent food processing tasks by adapting to different types of foodor process tasks.

These objectives are achieved by a heat treatment monitoring systemaccording to the appended claims.

In particular, to capture image from a heat treatment chamber (oven) itis advantageous to use an illumination in combination with outsidewindow tinting or darkening. This provides less impact of outside lightto the image processing of the oven inside pictures. It is recommendedto tint the window by at least 40%.

For industrial food processing it is advantageous to use a laser linegenerator, or any other light source, and a camera sensor, or any otheroptical sensor, to grasp information about the food being processed.With a procedure, also known as laser triangulation, a laser line may beprojected onto a measurement object, in order to obtain itscharacteristics.

Moreover, it is advantageous that the heat treatment of food isautomated such that no further human interaction is necessary besidesloading and unloading the oven or the heat treatment machine. However,even this step may be automated, if desired. In order to do so the heattreatment machine needs a treatment chamber that is camera monitored andequipped with an inside treatment chamber temperature sensor such as athermometer. Instead of using a camera an array of at least twophotodiodes may also be used. It is advantageous to use more sensorsacquiring signals related to inside treatment chamber humidity, time,ventilation, heat distribution, load volume, load distribution, loadweight, temperature of food surface, and interior temperature of thetreated food. The following sensors may as well be applied: hygrometer,laser triangulation, insertion temperature sensors, acoustic sensors,scales, timers, and many more. Further, cooling systems attached to anyheat sensible sensor applied may be applied. For instance, this may bean electrical, air or water cooling system such as a Peltier cooler orventilator, a thermoelectric heat pump, or a vapor-compressionrefrigeration, and many more.

Further it is advantageous that in a heat treatment process of food andin particular of baked goods with a heat treatment machine, such as anoven with heat treatment chamber, the inside temperature and theinterior camera image or other sensors can be used for the control ofpower supply or treatment parameters. According to the invention, thecamera image is suitable for the detection of parameters related to thechanging volume and/or the color of the food during heating of these.According to a model machine learned or fixed prior to this, it can bedetermined with this method for the heat treatment machine, if thetreated food is in a predefined desired process state, and with aclosed-loop control of the power of the heat treatment process theprocess may be individually adjusted. The desired process result may bereached at several locally distributed heat treatment machines bydistributing the parameters defined by the desired process conditions ofthe treated food. Moreover, the sensors used and the derived processdata, in particular the camera image, may be used to determine the typeand quantity of the food based on the data characteristics and thus tostart appropriate process variants automatically.

According to an embodiment of the present invention, a heat treatmentmonitoring system comprises: a heat treatment machine comprising a heattreatment chamber, a double glass window comprising an inside window andan outside window, and an illumination apparatus for illuminating theinside of the heat treatment chamber, and a monitoring apparatus mountedto the heat treatment machine and comprising a camera to observe theinside of the heat treatment chamber through the inside window, whereinthe visible transmittance of the outside window is lower than thevisible transmittance of the inside window to reduce reflections withinthe double glass window structure and outside illumination effects onimage processing of images recorded by the camera. Preferably, theoutside window is darkened by a coating. Preferably, a metal foil or atinting foil is applied at the outside window. Preferably, the outsidewindow comprises a tinted glass. Preferably, the outside window has amaximum visible transmittance of 60% Preferably, the double glass windowis a heat treatment machine door window of a heat treatment machine doorof the heat treatment machine. Preferably, the monitoring apparatus isadapted to generate high dynamic range (HDR) processed images of thefood to be heated within the heat treatment chamber. Preferably, themonitoring apparatus further comprises a casing and a camera sensormount, to which the camera is mounted. Preferably, the casing isequipped with heat sinks and fans to provide cooling of the camera.Preferably, the heat treatment machine is a convection or a deck ovenhaving at least two trays arranged in a stacked manner. Preferably, thecamera is tilted in such a way in a horizontal and/or a verticaldirection with regard to the double glass window to be adapted toobserve at least two trays at once in the convection or deck oven.Preferably, the heat treatment monitoring system comprises at least twocameras to observe each tray separately. Preferably, the heat treatmentmonitoring system further comprises a control unit being adapted toprocess and classify the images of food observed by the camera based ontraining data for determining an end time of a heating process for thefood. Preferably, the control unit is adapted to stop the heating of theheat treatment machine when the heating process has to be ended.Preferably, the control unit is adapted to open automatically the heattreatment machine door when the baking process has to be ended, orwherein the control unit is adapted to ventilate the heat treatmentchamber with cool air or air when the heating process has to be ended.

According to another embodiment of the present invention, a heattreatment monitoring system comprises a sensor unit having at least onesensor to determine current sensor data of food being heated; aprocessing unit to determine current feature data from the currentsensor data; and a monitoring unit adapted to determine a currentheating process state in a current heating process of the monitored foodby comparing the current feature data with reference feature data of areference heating process. Preferably, the heat treatment monitoringsystem further comprises a learning unit adapted to determine a mappingof current sensor data to current feature data and/or to determinereference feature data of a reference heating process based on featuredata of at least one training heating process. Preferably, the learningunit is adapted to determine a mapping of current sensor data to currentfeature data by means of a variance analysis of at least one trainingheating process to reduce the dimensionality of the current sensor data.Preferably, the learning unit is adapted to determine a mapping ofcurrent feature data to feature data by means of a variance analysis ofat least one training heating process to reduce the dimensionality ofthe current feature data. Preferably, the variance analysis comprises atleast one of principal component analysis (PCA), isometric featuremapping (ISOMAP) or linear Discriminant analysis (LDA) or adimensionality reduction technique. Preferably, the learning unit isadapted to determine reference feature data of a reference heatingprocess by combining predetermined feature data of a heating programwith a training set of feature data of at least one training heatingprocess being classified as being part of the training set by an userpreference. Preferably, the heat treatment monitoring system furthercomprises a recording unit to record current feature data of a currentheating process, wherein the learning unit is adapted to receive therecorded feature data from the recording unit to be used as feature dataof a training heating process. Preferably, the sensor unit comprises acamera recording a pixel image of food being heated, wherein the currentsensor data of the camera corresponds to the current pixel data of acurrent pixel image. Preferably, the current pixel data comprises firstpixel data corresponding to a first color, second pixel datacorresponding to a second color, and third pixel data corresponding to athird color. Preferably, the first, second and third color correspondsto R, G and B, respectively. Preferably, the camera is adapted togenerate HDR processed pixel images as current pixel data. Preferably,the heat treatment monitoring system further comprises a classificationunit adapted to classify the type of food to be heated and to choose areference heating process corresponding to the determined type of food.Preferably, the heat treatment monitoring system further comprises acontrol unit adapted to change a heating process from a proofing processto a baking process based on a comparison of the current heating processstate determined by the monitoring unit with a predetermined heatingprocess state. Preferably, the heat treatment monitoring system furthercomprises a control unit adapted to control a display unit being adaptedto indicate a remaining time of the heating process based on acomparison of the current heating process state determined by themonitoring unit with a predetermined heating process state correspondingto an end point of heating and/or to display images of the inside of theheat treatment chamber. Preferably, the heat treatment monitoring systemfurther comprises a control unit adapted to alert a user, when theheating process has to be ended. Preferably, the heat treatmentmonitoring system further comprises a control unit adapted to control atemperature control of a heating chamber, means to adapt humidity in theheat treatment chamber by adding water or steam, a control of theventilating mechanism, means for adapting the fan speed, means foradapting the differential pressure between the heat treatment chamberand the respective environment, means for setting a time dependenttemperature curve within the heat treatment chamber, means forperforming and adapting different heat treatment procedures likeproofing or baking, means for adapting internal gas flow profiles withinthe heat treatment chamber, means for adapting electromagnetic and soundemission intensity of respective electromagnetic or sound emitters forprobing or observing properties of the food to be heated. Preferably,the at least one sensor of the sensor unit comprises at least one ofhygrometer, insertion temperature sensor, treatment chamber temperaturesensor, acoustic sensors, scales, timer, camera, image sensor, array ofphotodiodes, a gas analyser of the gas inside the treatment chamber,means for determining temperature profiles of insertion temperaturesensors, means for determining electromagnetic or acoustic processemissions of the food to be treated like light or sound being reflectedor emitted in response to light or sound emitters or sources, means fordetermining results from 3D measurements of the food to be heatedincluding 3D or stereo camera systems or radar, or means for determiningthe type or constitution or pattern or optical characteristics or volumeor the mass of the food to be treated.

According to another embodiment of the present invention, a heattreatment monitoring system is provided, comprising: a heat treatment orbaking unit for baking or proofing goods or food to be heated or a foodprocessing line; a laser light distribution unit for generating a firstlaser beam and a second laser beam and for directing the first laserbeam and the second laser beam to a position of baking goods within thebaking unit; a first light detection unit for detecting the reflectionof the first laser beam scattered from the baking goods; a second lightdetection unit for detecting the reflection of the second laser beamscattered from the baking goods; a measurement unit for determining aheight profile of the baking goods according to the detections of thefirst light detection unit and the second detection unit; and a movingunit for changing a distance between the laser light distribution unitand the baking goods. Herein, the laser light distribution unitpreferably comprises: a first laser light generating unit for generatingthe first laser beam; and a second laser light generating unit forgenerating the second laser beam. Further, the laser light distributionunit preferably comprises: a primary laser light generating unit forgenerating a primary laser beam; an optical unit for generating thefirst laser beam and the second laser beam from the primary laser beam.The optical unit preferably comprises: a movable and rotatable mirror,towards which the primary laser beam is directed, for generating thefirst laser beam and the second laser beam alternately by moving androtating with respect to the primary laser light generating unit. Theoptical unit preferably comprises: a semi-transparent mirror, towardswhich the primary laser beam is directed, for generating the first laserbeam and a secondary laser beam; and a mirror, towards which thesecondary laser beam is directed, for generating the second laser beam.The first laser beam is preferably directed towards a first position;the second laser beam is preferably directed towards a second position;a piece of baking good is preferably moved from the first position tothe second position by the moving unit; and a change of the heightprofile of the piece of baking good is preferably determined by themeasurement unit. Preferably, the first laser beam is directed to afirst end of a piece of baking good and has an inclination of less than45° with respect to a support of the piece of baking good; the secondlaser beam is directed to a second end of the piece of baking goodopposite to the first end and has an inclination of less than 45° withrespect to the support; and the minimum angle between the first laserbeam and the second laser beam is greater than 90°. Preferably, themoving unit is a conveyor belt that moves the baking goods through thebaking unit. Preferably, the laser light distribution unit is locatedwithin the baking unit; and the first and second laser beams aredirected directly from the laser light distribution unit towards thebaking goods. Preferably, the laser light generating units are locatedoutside the baking unit; and the laser beams are directed towards thebaking goods by deflection mirrors. Preferably, the light detectionunits are located outside the baking unit; and the reflection of thelaser beams is guided to the light detection units by guiding mirrors.Preferably, the mirrors are heated. Preferably, the first and secondlaser beams are fan shaped; and the reflection of the first and secondlaser beams are focussed on the first and second light detection unitsby lenses. Preferably, the optical system constituted by the laser lightdistribution unit, the baking goods, and the light detection unitssatisfies the Scheimpflug principle. A method for monitoring baking ofthe present invention comprises the steps of: processing baking goods ina baking unit; moving the baking goods through the baking unit;generating a first laser beam and a second laser beam and directing thefirst laser beam and the second laser beam to a position of baking goodswithin the baking unit; detecting the reflection of the first laser beamscattered from the baking goods; detecting the reflection of the secondlaser beam scattered from the baking goods; and determining a heightprofile of the baking goods according to the detections of the scatteredfirst and second laser beams.

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIGS. 1A and 1B show a schematic cross sectional view and a schematicside view of an embodiment of a heat treatment monitoring system.

FIGS. 2A and 2B show the reflection properties of a conventional doubleglass window and a double glass window of an embodiment of a heattreatment monitoring system.

FIG. 3 shows different schematic views of another heat treatmentmonitoring system.

FIG. 4 shows a schematic view of an embodiment of an image sensor.

FIG. 5 shows a schematic view of another embodiment of an image sensor.

FIGS. 6A and 6B show a schematic front and side view of anotherembodiment of a heat treatment monitoring system.

FIG. 7 shows a schematic view of an embodiment of a heat treatmentchamber.

FIG. 8 shows a schematic view of an embodiment of a food productionsystem.

FIG. 9 shows a schematic view of an embodiment of a food productionsystem using laser triangulation.

FIG. 10 shows a schematic view of another embodiment of a foodproduction system using laser triangulation.

FIG. 11 shows a schematic top view of an embodiment of a tray withindication for arranging dough.

FIG. 12 shows a schematic view of an embodiment of a sensor systemintegrated in an oven rack.

FIG. 13 shows a schematic data processing flow of an embodiment of aheat treatment monitoring system.

FIG. 14 shows a cognitive perception-action loop for food productionmachines with sensors and actuators according to the present invention.

FIG. 15 shows categories of linear and nonlinear dimensionalityreduction techniques.

FIG. 16 shows a mapping of two-dimensional test data to athree-dimensional space with an optimal linear separator.

FIG. 17 shows an architecture according to the present invention andcomponent groups to design agents for process monitoring or closed-loopcontrol in food production systems using a black-box model with sensorsand actuators.

FIG. 18A shows a schematic cross sectional view of an embodiment of aheat treatment monitoring system.

FIG. 18B shows a block diagram of an embodiment of a heat treatmentmonitoring system.

FIGS. 1A and 1B illustrate a heat treatment monitoring system 100according to an embodiment of the present invention. FIG. 1A illustratesa schematic crosssectional top view of the heat treatment monitoringsystem 100, while FIG. 1B illustrates a schematic front view thereof.

As illustrated in FIGS. 1A and 1B the heat treatment monitoring systemor baking monitoring system or proofing and/or baking monitoring system100 has an oven 110 with a heat treatment or oven chamber 120, at leastone double glass window 130 at a side wall of the oven 110 and anillumination apparatus 140 inside the oven chamber 120.

The heat treatment machine or oven 110 may be any oven that may beconventionally used for cooking of food, in particular for baking orproofing of bread. The oven may cook food using different techniques.The oven may be a convection type oven or a radiation type oven.

The heat treatment or oven chamber 120 captures most of the interior ofthe oven 110. Inside the oven chamber 120 food is cooked. The food maybe placed on a differing number of trays which can be supported at theoven chamber walls. The food may also be placed on moveable carts withseveral trays, which can be moved inside the oven chamber 120. Insidethe oven chamber 120 a heat source is provided, which is used to cookthe food. Moreover, also a ventilation system may be comprised insidethe oven chamber to distribute the heat produced by the heat source moreevenly.

The inside of the oven or heat treatment chamber gets illuminated by anillumination apparatus 140. The illumination apparatus 140 may bearranged inside the oven or heat treatment chamber as shown in FIG. 1A.The illumination apparatus 140 may also be located outside the ovenchamber 120 and illuminate the oven chamber 120 through a window. Theillumination apparatus 140 may be any conventional light emittingdevice, e.g. a light bulb, a halogen lamp, a photodiode or a combinationof several of these devices. The illumination apparatus 140 may befocused on the food to be cooked inside the oven chamber 120. Inparticular, the illumination apparatus 140 may be adjusted or focusedsuch that there is a high contrast between the food to be cooked and thesurrounding interior of the oven chamber 120 or between the food andtray and/or carts on which the food is located. Such a high contrast maybe also supported or generated solely by using special colors for thelight emitted by the illumination apparatus 140.

In a wall of the oven chamber 120 a window is provided. In order toprevent a loss of heat out of the oven chamber 120, the window ispreferably a double glass window 130 having an outer glass pane oroutside window 135 and an inner glass pane or inside window 136. Thedouble glass window 130 may prevent heat dissipation between the insidewindow 136 and the outside window 135 by providing a special gas or avacuum between the inside window 136 and the outside window 135. Thedouble glass window 130 may also be cooled by air ventilation betweenthe inside window 136 and the outside window 135 to prevent a heating ofthe outside window 135, wherein no special gas or a vacuum is providedbetween the inside window 136 and the outside window 135. Theillumination apparatus 140 may be also be provided between the insidewindow 136 and the outside window 135. The outer glass surface of theoutside window 135 is less hot and thus suitable for mounting a camera160. It may be further beneficial to use an optical tunnel between theinside window 136 and the outside window 135, because this again reducesreflections and heat impact.

Through the double glass window 130 a cooking or baking procedure insidethe oven chamber 120 may be observed from outside the heat treatmentmachine or oven.

As is illustrated in FIG. 1B a monitoring apparatus 150 is mounted onthe heat treatment machine or oven 110. The monitoring apparatus 150 ismounted across the outside window 135 of the double glass window 130 andcomprises a camera 160 arranged next to the outside window 135, which isused to observe the food inside the oven chamber 120 during cooking orbaking. The camera 160 may be any conventional camera which is able toprovide image data in a computer accessible form. The camera 160 may forexample be charged coupled device (CCD) camera or a complementarymetal-oxide-semiconductor (CMOS) camera. The camera 160 obtains imagesof the cooked food during the cooking procedure. As will be describedbelow these images may be used for automatically controlling the cookingor baking procedure. Although the camera 160 is preferably mounted at anoutside of the outside window 135 to be easily integrated within themonitoring apparatus 150, wherein the camera 160 then observes an insideof the heat treatment chamber 120 through the double glass window 130,the camera 160 may also be provided between the inside window 136 andthe outside window 135 to observe an inside of the heat treatmentchamber through the inside window 136.

However, a problem arises if an external light source is present outsideof the oven chamber 120 in front of the double glass window 130.

As illustrated in FIG. 2A, irritating light 272 emitted by an externallight source 270 may pass through an outside window 235′ of a doubleglass window, but might be reflected by the inside window 236 into acamera 260 observing food 280 to be cooked. Therefore, the camera 260does not only obtain light 282 emitted or reflected from the food 280,but also the irritating light 272, reflected at the inside wall 236.This result in a deterioration of the image data provided from thecamera 260 and may therefore adversely affect an automatic bakingprocess.

In the present embodiment this adverse effect is prevented by hinderingthe irritating light to pass through an outside window 235. This may bedone by tinting or darkening the outside window 235. Then, theirritating light 272 is reflected or absorbed by the outside window 235and does not reach the inside window 236. Hence, no irritating light 272is reflected into the camera 260 by the inside window 236 and the camera260 captures only correct information about the food 280. Therefore,according to the present embodiment a deterioration of the automatedfood processing procedure is prevented by tinting or darkening theoutside window 235.

Thus, to capture images from the heat treatment chamber 120 of the oven110, it is advantageous to use an illumination apparatus 140 incombination with tinting or darkening of the outside window 235. Thisprovides less impact of outside light to the image processing of theoven inside pictures.

According to the present invention, the visible transmittance of theoutside window 135 is lower than the visible transmittance of the insidewindow 136. Herein, the visible transmittance of the outside window 135is lower than 95%, more preferably lower than 80%, and in particularlower than 60% of the visible transmittance of the inside window 136.Further, the outside window 235 of the double glass window 130 may havepreferably a maximum visible transmittance of 75%. The visibletransmittance is the transmittance of light being incident normal to theglass window surface within a visible wavelength range, i.e. between 380nm to 780 nm. It is further preferable to tint the window by at least40%, thus the maximum visible transmittance is 60%. In other words, atleast 40% of the incoming light is absorbed or reflected by the outsidewindow 235 and 60% of the light is transmitted through the outsidewindow 235. The inside window 236 may have a visible transmittance ofusual glass. It is further preferred to tint the window by at least 60%,leading to a transmittance of 40%. A darkening coating or foil may beapplied advantageously at the outside window of a double glass door ofthe oven to prevent deterioration of the coating due to thermal effects.Due to the darkening of the outside window, reflections of the lightcoming from an outside of the oven can be significantly reduced. Theoven door window can be darkened by a metal foil or coating (mirroredwindow) or by a tinting foil. The oven door window can be a tintedwindow comprising e.g. a tinted outside and/or inside glass. If thecamera is mounted on the outside window 135, the darkening orreflectivity of the outside window 135 at the location of the camera maybe spared, for example by having a hole within the coating to ensure anobservation of the camera through the hole in the coating of the outsidewindow 135, wherein the area of the hole is not included for thedetermination of the transmittance of the outside window 135.

The oven or heat treatment machine 110 may further comprise an oven dooror heat treatment machine door, by which the oven chamber 120 can beopened and closed. The oven door may comprise a window, through whichthe oven chamber 120 can be observed. Preferably, the window comprisesthe double glass window 130 for preventing thermal loss of the heatingenergy for the oven chamber 120. Thus, the heat treatment monitoringsystem 100 may comprise the monitoring apparatus 150 and the oven 110comprising the monitoring apparatus 150, or an oven 110 having themonitoring apparatus 150 mounted to its oven door.

Thus, also reflections within the double glass window structure of theoven door window can be reduced. Consequently, outside illuminationeffects on image processing are neglectable. Thus, with a respectiveillumination intensity of the oven chamber 120, the inside of the ovenchamber 120 may be observed by the camera 160 of the monitoringapparatus 150.

FIG. 3 shows different views of an embodiment of the heat treatmentmonitoring system illustrated in FIGS. 1A and 1B.

As illustrated in FIG. 3, a monitoring apparatus 350 is mounted to thefront side of a heat treatment machine or an deck oven 310 of a heattreatment monitoring system 300. The monitoring apparatus 350 comprisesa casing, a camera sensor mount, and a camera mounted to the camerasensor mount to observe an inside of a heat treatment or oven chamberthrough a heat treatment machine or oven door window 330. The camera istilted in such a way in a horizontal and/or a vertical direction withregard to the oven door window 330 to be adapted to observe at least twobaking trays at once in the deck oven 310.

According to another embodiment the sensor mounting and the casing arecooled with fans for the inside. Further as can be seen from FIGS. 4 and5 the camera sensor mount of the monitoring apparatus 350 may beequipped with heat sinks and fans to provide cooling. The sensor mountand the casing may be optimized to have an optimal viewing angle to seetwo baking trays at once in the oven.

FIGS. 6A and 6B show a top view and a side view of another embodiment ofthe heat treatment monitoring system illustrated in FIGS. 1A and 1B,respectively.

As illustrated in FIG. 6A a monitoring apparatus 650 is mounted on aheat treatment machine or an oven 610 of a heat treatment monitoringsystem 600. The monitoring apparatus 650 overlaps partially with adouble glass window 630 of a heat treatment machine or oven door 632.The monitoring apparatus 650 comprises a camera inside a casing.Moreover, the monitoring apparatus 650 comprises a display 655, whichallows information to be displayed to a user and enables a userinteraction.

The oven 610 may have a convection oven on top and two deck ovensunderneath as illustrated in FIGS. 6A and 6B.

Moreover, according to an embodiment the monitoring apparatus 150 maycomprise an alert device to inform the user when the baking process hasto be ended. In addition, the monitoring apparatus 150 may comprise acontrol output to stop, for example the heat treatment of the oven 110and/or to open automatically the oven door and/or to ventilate the ovenchamber 120 with cool air or air. The oven 110 and the monitoringapparatus 150 form together the heat treatment monitoring system 100.

According to a further embodiment, the monitoring apparatus 150 isadapted to generate high dynamic range (HDR) processed images of bakinggoods within the oven chamber 120. This is particularly advantageous incombination with the tinted outside window 135, since the lightintensity of the light coming from the baking chamber 120 inside isreduced by the tinting foil and the HDR processing enables bettersegmentation. Moreover, by using HDR processing a contrast betweenbaking goods and their surroundings like oven walls or trays may beenhanced. This enables the heat treatment monitoring system 100 todetermine a contour or shape of baking goods even more precisely.

FIG. 7 demonstrates a possible sensor setup for a treatment chamber 720according to a further embodiment. As before, the treatment chamber 720is monitored with at least one camera 760. The camera 760 may alsocomprise an image sensor or a photodiode array with at least twophotodiodes. It is advantageous to use more than one camera in order tomonitor several trays that may be loaded differently. At least onecamera 760 may be positioned within the treatment chamber 720 but it isadvantageous to apply a window that reduces the heat influence towardsthe camera(s) 760, in particular a double glass window 730. The doubleglass window 730 may be in any wall of the treatment chamber.

As described above it is advantageous to apply illumination to thetreatment chamber 720 by integrating at least one illumination apparatusas e.g. a bulb or a light-emitting diode (LED). Defined treatmentchamber illumination supports taking robust camera images. It is furtheradvantageous to apply illumination for at least one specific wavelengthand to apply an appropriate wavelength filter for the camera or imagesensor or photodiode array 760. This further increases the robustness ofthe visual monitoring system. If the wavelength is chosen to be infraredor near-infrared and the image sensor 760 and optional filters arechosen accordingly, the visual monitoring system may gather informationrelated with temperature distribution that may be critical for certainfood treatment processes.

The camera or visual system 760 may be equipped with a specific lenssystem that is optimizing the food visualization. It is not necessary tocapture images related to all loaded food, as the processing state of aload is very similar among the load itself. Further it may be equippedwith an autofocus system and brightness optimization techniques. It isadvantageous to use several image sensors 760 for specific wavelengthsin order to gather information about changes in color related to thefood treatment. It is advantageous to position the camera or imagesensors 760 to gather information of volume change of the food duringheat treatment. It may be in particular advantageous to setup a top-viewof the food products.

It may also be advantageous to attach a second oven door or treatmentchamber opening to a pre-existing opening system. The sensor system orin particular the camera, and the illumination unit may then be positionat the height of the oven door window. This door on top of a door ordouble door system could be applied if the sensor system is retrofittedto an oven.

Each of the monitoring apparatuses described above may be mounted to thefront side of an oven, as can be seen for example in FIGS. 1A, 1B, 3,4A, and 4B. The monitoring apparatus comprises a casing, a camera sensormount, and a camera mounted to the camera sensor mount to observe aninside of an oven chamber through an oven door window. The camera istilted in such a way in a horizontal and/or a vertical direction withregard to the oven door window to be adapted to observe at least twobaking trays at once in the deck oven. The monitoring apparatus mayfurther comprise an alert device to inform the user when the bakingprocess has to be ended. In addition, the monitoring apparatus maycomprise a control output to stop, for example the heating of the ovenand/or to open automatically the oven door and/or to ventilate the ovenchamber with cool air or air. The oven and the monitoring apparatus formtogether a heat treatment monitoring system.

As discussed above one camera sensor is used to observe the bakingprocesses. According to another embodiment it is beneficial to useseveral camera sensors. If every tray within a heat treatment chamberhas at least one camera sensor aligned, a monitoring and controlsoftware may gain information for every tray individually. Thus, it ispossible to calculate a remaining baking time for every tray.

The remaining baking time may be used to alert the oven user to open thedoor and take out at least one of the trays, if the baking time hasended before the other trays. According to the invention it is possibleto alert the user by means of a remote or information technology system.The alert may happen on a website display, on a smart phone, or on aflashlight next to the counter. This has the advantage that the user isbeing alerted at their usual working place that may be not in front ofthe oven.

According to another embodiment of the monitoring system of the presentinvention the monitoring system may be used in industrial foodproduction systems, e.g. in baking or pre-baking lines or in doughpreparation systems that form and portion dough. However, the monitoringsystem may also be used in any other area of food production orprocessing.

FIG. 8 illustrates a monitoring system 800 with at least one sensorsystem setup 850, for heat treatment machines or ovens 810 (bakingunits) with belt conveyor 815 (moving unit). These ovens 810 are usuallyused in industrial food production systems.

The sensor system 850 may have at least one sensor of the following:hygrometer, insertion temperature sensor, treatment chamber temperaturesensor, acoustic sensors, laser triangulation, scales, timer, camera,image sensor, array of photodiodes. Part of this sensor system 850 isalso the supporting devices such as illumination or cooling or movementalgorithms.

According to an embodiment laser triangulation may be used to acquireinformation regarding a food volume. Then the sensor system setup 850comprises a laser light distribution unit, which generates and directslaser beams towards baking goods within the oven or baking unit 810. Thelaser light distribution unit may direct the laser beams on a singlepiece of baking good at the same time or, according to anotherembodiment at least twice within the food treatment process to acquireinformation regarding the change of volume over time.

The volume information and/or a height profile of the baking good isthen acquired by a measuring unit, which analyses detection results oflight detection units, which detect the reflection of the laser beamsfrom the baking goods. There may be a single or several light detectionunits for all laser beams or one light detection unit for each laserbeam.

According to another embodiment at least one additional sensor system852 may be placed at different positions inside or outside of the heattreatment machine. Alternatively, the sensor system 850 may be appliedat a position where the belt conveyor passes the food twice at differenttimes of processing. Alternatively, the sensor system 850 may move withthe same speed as the belt conveyor 815.

According to further embodiments more than one camera sensor or opticalsensor and more than one laser line generator for laser triangulationmay be used. According to an embodiment illustrated in FIG. 9, amonitoring system 900 comprises at least two monitoring apparatuses eachwith a laser line generator 955 and a light receiving device 960 as e.g.a camera or a photo diode array. Thus, a laser light distribution unitaccording to this embodiment comprises a first laser light generatingunit and a second laser light generating unit.

From the laser light generators 955 laser beams 956 are emitted towardsfood 980 as e.g. raw or pre-baked dough on a belt conveyor 915. From thefood 980 the laser beams are reflected towards the light receivingdevices 960. As the position of the laser light generators 955 and thelight receiving devices 960 with respect to each other and with respectto the belt conveyor 915 is known, the distance of the laser lightgenerators 955 to the food 980 can be obtained by triangulation from theexact position at which the laser beams 956 are observed within thelight receiving devices 960. Hence, using such laser triangulation thesurface profile of processed food 980 may be determined.

As is shown in FIG. 9 the laser beams 956 are directed directly towardsthe food or baking goods 980 and are scattered directly towards thelight receiving devices or light detection units 960. According toanother embodiment the light paths of the laser beams may be altered bydeflection or guiding mirrors. Then, the laser light generators 955 orthe light detection units 960 may be located also outside of the heattreatment chamber or baking unit. This allows for a more flexible designof the heat treatment monitoring system. Moreover, in order to preventsteaming up of the mirrors, these may be heated to a temperaturesufficiently high to hinder the steaming up, but low enough to notdamage the mirrors.

As shown in FIG. 9 the laser beams 956 from the laser light generators955 are focused such that food 980 at different stages of production ismonitored. Note that although in FIG. 9 it is shown that the laser lightgenerators 955 focus on two neighboring pieces of food 980, they mayjust as well focus on pieces of food 980 which are a greater distanceapart from each other. For example, the two pieces of food may beseparated by several meters or the laser light generators 955 may belocated at an entrance and the exit of a baking chamber through whichthe belt conveyor 915 runs and observe the surface profile of food 980during entry and exit of the baking chamber. To this end, the laserlight generators or generation units 955 may also be arranged such thatthey emit light nearly perpendicular from the top towards the food 980.

Note also that the laser line generators 955 do not need to be locatedabove the belt conveyor 915, but may also be located at a side of thebelt conveyor 915. Of course the at least two laser line generators 955may also be located at different sides of the belt conveyor 915.

Hence, by using two or more laser light generators 955 that focus ondifferent pieces of food 980 and observe the respective surfacestructure of the pieces of food 980 a difference in this surfacestructure caused by the baking or food production process may beobserved, as the belt conveyor or moving unit 915 moves food 980 throughthe baking unit from a focus point of a first laser beam towards a focuspoint of a second laser beam. This information about the difference insurface structure at various stages of the baking or food productionprocess may be used to automatically control the process and henceallows for automated baking or food production.

The laser beams 956 may be dot-like or may be fan-shaped and extendacross the whole width of the belt of the belt conveyor 915. By usingfan-shaped laser beams 956 a three dimensional profile of the food 980running on the belt conveyor 915 may be obtained that may serve evenbetter for automatically controlling the baking or food productionprocess. Then, the reflection of the fan shaped laser beams from thefood may be collimated or concentrated by lenses on the light detectionunits 960, in order to allow for small light detection units 960 whichcan be easily integrated in the heat treatment monitoring system.

As shown in FIG. 10 in addition to observing different pieces of food itis especially beneficial to align at least two sensor systems of amonitoring system 1000 per piece of food in a 45 degree tilted angle,observing the measurement objects 1080 from top left and top right. Thisis advantageous because when observing round objects, laser lightgenerators 1055 and their respectively aligned light receiving devices1060 can measure the surface structure of the round objects in areas,that may have been obscured when only using one sensor from a top viewwould have been used. According to another embodiment the laser beamsmay be inclined even less than 45° with respect to the conveyor belt orthe tray, which supports the food 1080. Then, the surface structure nearthe support of the food may be observed even better.

In case that fan shaped laser beams are used, the inclination of theplanes spanned by the fans should be less than 45° with respect to thesupport of the food 1080. This also means that the angle between thelaser beams has to be greater than 90°.

Note that although in FIG. 10 it is shown that the laser lightgenerators 1055 focus on the same piece of food 1080, they may just aswell focus on two different pieces of food 1080, which are separatedfrom each other. For example, the two pieces of food may be separated byseveral meters or the laser light generators 1055 may be located at anentrance and the exit of a baking chamber through which the beltconveyor runs and observe the surface profile of food 1080 during entryand exit of the baking chamber.

Note also that the laser line generators 1055 do not need to be locatedabove the belt conveyor, but may also be located at a side of the beltconveyor. Of course the at laser line generators 1055 may also belocated at different sides of the belt conveyor.

Furthermore according to another embodiment there may be a lasertriangulation display within the oven. Then, at least two lasertriangulation sensors and two line lasers, looking at the baked productsfrom approximately 45 degree angle (top left and top right) may be used.This gives the advantage that one can measure also the rounding of thebaked products at their bottom, while by using one laser line and camerafrom top view, the bottom half rounding is obscured and not accountedfor in the measurements.

Hence, according to these embodiments additional information about thebaking or food production process may be provided based on whichautomated baking or food production may be performed more efficient andreliable.

According to another embodiment a laser line generator, or any otherlight source, and a camera sensor, or any other optical sensor, may beused to grasp information about the food being processed. With theprocedure described above, also known as laser triangulation, a laserline may be projected onto a measurement object. An optical sensor, asensor array or typically a camera can be directed towards thismeasurement object. If the camera perspective or the viewing point andthe respective plane and the plane of the laser line generator, formedby the light source and the ends of the projected laser line, are notparallel or are at an angle, the detected optical information may beused to perform measurements providing information about size and shapesincluding a three dimensional structure or volume.

In the embodiments described above two laser light generating units havebeen used in order to generate and direct the laser beams. According toanother embodiment a primary laser light generating unit may be used togenerate a primary laser beam, which is then distributed by an opticalunit within the baking unit. Using such structure within the heattreatment monitoring system makes it possible to save energy costs andspace by reducing the number of laser light generating units.

Moreover, the laser light generating unit may be located outside of thebaking unit and only the primary laser beam may be input into the bakingunit. This makes it possible to choose a structure of the heat treatmentmonitoring system more flexibly, especially if also the light detectionunits are provided outside of the baking unit.

The optical unit may be any type of optical system that allows forsplitting of a single primary laser beam into two or more laser beams.For example, the optical system may comprise a semi-transparent mirror,which reflects a part of the primary laser beam towards a first positionto be observed and transmits a part of the primary laser beam toward amirror, which reflects the light towards a second position of interest.The primary laser beam may also be separated such that its parts aredirectly directed towards the positions to be observed. According toanother embodiment there may also be more mirrors and/or lenses withinthe light path of the primary laser beam.

According to another embodiment the optical unit may comprise a movableand rotatable mirror, which generates laser beams alternately. To thisend the moveable and rotatable mirror may be provided above the food orbaking goods and may be moved and rotated such that the primary laserbeam is directed to different pieces of food or different positions on asingle piece of food at different times. Hence, volume informationcollected by the measurement unit will refer to different positionswithin the baking unit according to time.

Using such mirrors reduces the space requirements within the baking unitand allows for a flexible design of the heat treatment monitoringsystem. Moreover, a user may switch operation easily from a mode, inwhich two different pieces of food are observed, in order to obtaininformation about the change of the height profile and/or volume profileof the food, and a mode, in which a single piece of food is observedfrom different directions, in order to obtain the overallthree-dimensional shape of the piece of food also near the support ofthe piece of food.

The movable and rotatable mirror may also perform such different tasksin parallel.

Of course also the mirrors used in connection with a primary laser beammay be heated in order to prevent steam up.

According to another embodiment, the optical system constituted by thelaser light distribution unit, the food or baking good, and the lightdetection unit may satisfy the Scheimpflug principle. This guaranteesthat the image of the baking good sampled by the laser beams is alwaysfocused on the light detection unit, and allows therefore for an exactmeasurement of a height profile of the baking good.

According to another embodiment laser triangulation may be combined withgrey image processing to gather simultaneous information about shape andsize as well as information about texture, colour and other opticalfeatures. The resulting process data may be used to generate uniquefeatures for the measurement object, in this case food. This may beshape, size, volume, colour, browning, texture, pore size and density offood being processed such as dough or baked bread, which may be sliced.Some or all of the named information may be used to interpret the sensordata, in order to allow for automated baking or food processing

In the embodiments described above the data capturing is performedmainly by image sensors such as cameras or photo diode arrays. However,according to further embodiments the data obtained by the image sensorsmay be supplemented with data from a variety of other sensors such ase.g. hygrometers, insertion temperature sensors, treatment chambertemperature sensors, acoustic sensors, laser, scales, and timers.Furthermore, a gas analyser of the gas inside the treatment chamber,means for determining temperature profiles of insertion temperaturesensors, means for determining electromagnetic or acoustic processemissions of the food to be treated like light or sound being reflectedor emitted in response to light or sound sources, means for determiningresults from 3D measurements of the food to be heated including 3D orstereo camera systems or radar, means for determining the type orconstitution or pattern or optical characteristics or volume or the massof the food to be treated can be also used as sensors for the sensorunit 1810 as described below. Automated food processing or baking maythen be controlled based on all data from all sensors.

For example, referring back to FIG. 7, the treatment chamber 720 may befurther equipped with at least one temperature sensor or thermometer762. Although this is only illustrated within FIG. 7 any otherembodiment described herein may also comprise such a temperature sensor762. When treating food with heat, temperature information relates toprocess characteristics. It may contain information towards heatdevelopment over time and its distribution inside the treatment chamber.It may also gather information about the state of the oven, its heattreatment system and optional pre-heating.

It may also be advantageous to integrate insertion thermometers.Insertion thermometers enable to gather inside food temperatureinformation that is critical to determine the food processing state. Itis advantageous in bread baking to acquire information related to theinside and crumb temperature.

Moreover, a color change progress in time of the food to be heated maybe used to determine an actual temperature within the oven chamber andmay be further used for a respective temperature control in the bakingprocess. The treatment chamber 720 or any other embodiment describedherein may be equipped with at least one sensor related to treatmentchamber humidity such as a hygrometer 764. In particular for breadbaking gathering information related to humidity is advantageous. Whenthe dough is heated the containing water evaporates resulting in adifference in inside treatment chamber humidity. For instance, with aircirculation the treatment chamber humidity during a baking process mayfirst rise and then fall indicating the food processing state.

The treatment chamber 720 or any other embodiment described herein mayfurther be equipped with at least one sensor gathering information ofthe loaded food weight and eventually its distribution. This may beaccomplished by integrating scales 766 in a tray mounting system of theheat treatment chamber 720. The tray mounting or stack mounting may besupported by rotatable wheels or discs easing the loading of the oven.The scales 766 could be integrated with the wheels or discs and takethem as transducer. It is advantageous to acquire the weight informationfor every used tray or set of trays individually in order to haveinformation related about the total food weight and its relativedistribution as the desired energy supply and its direction during theheat treatment may vary significantly. Further it is advantageous toacquire information of the food weight differences over time whiletreating it with heat. For instance in bread baking, the dough roughlyloses around 10% of its initial weight. Further, it is possible toacquire information regarding the state of dough or food by emission andcapturing of sound signals, e.g. by a loudspeaker and microphone 768.

Moreover, in the described embodiments alternative cameras or imagesensors or photodiode array sensors and eventually alternativeillumination setups may be used. Instead of placing the camera behind awindow on any treatment chamber wall, it or a second camera may as wellbe integrated with the oven door or treatment chamber opening.

Instead of integrating illumination into any treatment chamber wall, itmay as well be integrated into the oven door or treatment chamberopening. Commonly ovens door have windows to enable human operators tovisually see the food treated and to supervise the process. According toanother embodiment at least one camera or image sensor or photodiodearray or any other imaging device may be integrated into an oven door ora treatment chamber opening. An oven door without window for humanoperators may be designed more energy efficient as heat isolation may bebetter. Further, differences in outside lightening do not influence withthe treatment chamber monitoring camera images that would then only relyon the defined treatment chamber illumination. However, one should notethat such a setup might not be easily installed later on an alreadyexisting oven.

Further, it may be advantageous to integrate a screen or digital visualdisplay on the outside wall of the oven door or at any other placeoutside of the treatment chamber. This screen may show images capturedfrom the treatment chamber monitoring camera. This enables a humanoperator to visually supervise the baking process, although it is anobject of the invention to make this unnecessary.

Further, it may be advantageous to use trays or a stack of trays thatindicates the food distribution. For instance, in bread baking, whenloading the oven the dough placement may vary for every baking cycle.These differences can be coped with by image processing with matchingand recognition techniques. It is advantageous to have a similar loadingor food placement for every production cycle as indicated in FIG. 11. Anautomated placement system may be applied when setting trays 1100. Formanual placements at least some of the used trays may have indication1110 of where to place the dough. As indication bumps, dumps, pans,molds, food icons, food drawings, or lines may be used.

Moreover, when integrating a camera sensor in an oven environment or afood processing system it may be of advantage to integrate coolingdevices. These may be at least one cooling plate, at least one fanand/or at least one water cooling system.

Further, a shutter may be used, that only exposes the camera sensor whennecessary. It may often not be necessary to take many pictures and itmay often be feasible to only take pictures every 5 seconds or less. Ifthe shutter only opens every 5 seconds the heat impact on the camerachip is significantly lower, which reduces the possibility of an errordue to a heat impact and thus increases the reliability of the heattreatment monitoring system.

It may be further of advantage to take at least two pictures or more ortake one exposure with several non-destructive read outs and combine thepixel values. Combining may be to take a mean or to calculate onepicture out of at least two by means of High Dynamic Range Imaging. Incombination with a shutter or stand alone it is possible to applywavelength filters, that let only relevant wavelengths pass, forinstance visible light or infrared radiation. This may further reducethe heat impact on the camera chip and hence increase the reliability ofthe monitoring system even further.

In another embodiment, illustrated in FIG. 12, a sensor systemintegration for oven racks or moving carts used in some oven designs maybe used. For rotating rack ovens, the sensor system may be integratedinto the oven rack as demonstrated with 1200. The sensor system isintegrated above at least one of the food carrying trays. The sensorsystem in the cart may have at least one sensor of the following:hygrometer, insertion temperature sensor, treatment chamber temperaturesensor, acoustic sensors, scales, timer, camera, image sensor, array ofphotodiodes. Part of the rack integrated sensor system is alsosupporting devices such as illumination or cooling as demonstrated inthis invention. It further is object of the invention to have anelectrical connection such as a wire or electrical plugs at the mountingof the rack as demonstrated with 1210. It is further advantageous tointegrate at least part of the sensor system into the rotating rack ovenwall as demonstrated with 1220. This is advantageous to reduce the heateffects onto the sensor system. For the camera, image sensor, orphotodiode array it is advantageous to apply an image rotation ormovement correction algorithm in order to correct the rack rotation orfood movement. This algorithm may be supported by a measured or pre-setparameter from the oven control regarding the rotation or movementspeed.

In another embodiment a graphical user interface (GUI) may show picturesof every tray and deck within an oven. In a convection oven the end timefor every tray may be determined separately. This means that if one trayis finished earlier than another, the user may get a signal to emptythis tray and leave the others in. This is advantageous because manyovens may not have equal results for different trays. Moreover, one maybake different products on each tray, if they have approximately thesame baking temperature. Hence, it is possible to operate a single ovenmore flexible and efficient.

In another embodiment the oven may also determines the distribution ofthe baked goods on a tray. An oven may also reject poorly loaded trays.

Using one or several of the sensors described above data about thebaking or food processing procedure may be collected. In order to allowfor an efficient and reliable automated baking or food processing theprocessing machines such as ovens or belt conveyors need to learn how toextract relevant data from all data, how to classify the processed foodand the stage of food processing based on these data, and how toautomatically control the processing based on the data and theclassification. This may be achieved by a heat treatment monitoringsystem that is able to control a baking process based on machinelearning techniques.

FIG. 13 demonstrates a control unit and a data processing diagramaccording to which the data of any of the aforementioned embodiments maybe handled.

Here, the control unit or heat treatment monitoring system 1300, for theheat treatment machine 1310, recognizes the food to be processed withany of the described sensor systems. The recognition of the food to beprocessed may be accomplished with the unique sensor data input matrixD_(a). This sensor data input matrix or a reduced representation of itcan be used to identify a food treatment process with its datacharacteristic or data fingerprint.

The control unit 1300 has access to a database that enables to comparethe sensor data input matrix with previously stored information,indicated with 1301. This enables the control unit 1300 to choose acontrol program or processing procedure for the present food treatment.Part of this procedure is according to an embodiment a mapping X_(c) ofthe sensor data input matrix D_(a) to an actuator control data matrixD_(b),

D _(a) X _(c) =D _(b).  (Formula 1.00)

With the actuator control data matrix D_(b) the heat treatment machine1310 controls the food processing, for instance by controlling ovencontrol parameters such as energy supply or start and end time ofprocessing. The heat treatment machine then operates in a closed-loopcontrol mode. Typically, the sensor data input matrix D_(a) issignificantly higher in dimension compared to the actuator control datamatrix D_(b).

According to an embodiment it is advantageous to find a mapping X_(c) aswell as a reduced representation of the sensor data input matrix D_(a)with methods known from machine learning. This is because the type offood to be processed and the according procedures are usuallyindividually different.

From a data processing point of view the relations between sensor datainput and appropriate actuator output may be highly non-linear and timedependent. Today these parameters are chosen by human operators commonlywith significant know how in a time consuming configuration of the heattreatment machine. According to an embodiment of the present inventionwith initial data sets learned from a human operator, machine learningmethods can perform the future system configuration and expediteconfiguration times as well as increase processing efficiency as well asquality.

All applied data may be stored in databases. According to the inventionit is beneficial to connect the heat treatment machine with a network.With the means of this network, any database data may be exchanged. Thisenables a human operator to interact with several locally distributedheat treatment machines. In order to do so the heat treatment machinehas equipment to interact with a network and use certain protocols suchas Transmission Control Protocol (TCP) and Internet Protocol (IP).According to the invention the heat treatment machine can be equippedwith network devices for a local area network (LAN) a wireless areanetwork (WLAN) or a mobile network access used in mobiletelecommunication.

In any of the previously described embodiment a baking or foodprocessing procedure may contain a learning phase and a productionphase. In the learning phase a human operator puts food into the heattreatment machine. It is treated with heat as desired by the humanoperator. This can be carried out with and without pre-heating of theheat treatment chamber. After the processing with heat the humanoperator may specify the type of food and when the desired process statehas been reached. The human operator can also provide information whenthe product was under baked, over baked and at desired process state.

Using the described machine learning methods the machine calculates theprocessing parameters for future food production. Then the heattreatment machine or heat treatment machines in a connected network canbe used to have additional learning phases or go into automatedproduction. When in automated production the human operator just putsthe food into the heat treatment machine with optional pre-heating. Themachine then detects the food in the treatment chamber and performs thepreviously learned heat treatment procedure.

When the desired food process state has been reached or simply, when thebread is done, the machine ends the heat treatment process. It can do soby opening the door or end the energy supply or ventilate the hot airout of the treatment chamber. It can also give the human operator avisual or acoustical signal. Further, the heat treatment machine may askfor feedback from the human operator. It may ask to pick a category suchas under baked, good, or over baked. An automated loading system thatloads and unloads the treatment chamber may fully automate theprocedure. For this purpose a robotic arm or a convection belt may beused.

Recent techniques in machine learning and the control of food processinghave been examined to create adaptive monitoring. Artificial NeuralNetworks (ANN), Support Vector Machines (SVM), and the Fuzzy K-NearestNeighbor (KNN) classification have been investigated as they apply tospecial applications for food processing. One aim of the presentinvention is to evaluate what machine learning can accomplish without aprocess model defined by a human operator.

In the following, a brief overview of the theories underlying thepresent invention is given. This includes techniques for reducing sensordata with dimensionality reduction, such as Principal ComponentAnalysis, Linear Discriminant Analysis, and Isometric Feature Mapping.It also includes an introduction of classification and supervised aswell as unsupervised learning methods such as Fuzzy K-Nearest Neighbor,Artificial Neural Networks, Support Vector Machines, and reinforcementlearning. For the number format, the thousand separator is a comma “,”and the decimal separator is a point “.”; thus, one-thousand isrepresented by the number 1,000.00.

Feature Extraction and Dimensionality Reduction

The present invention does not seek nor desire to achieve human-likebehavior in machines. However, the investigation of something likecognitive capabilities within food processing or production machines ofartificial agents capable of managing food processing tasks may providean application scenario for some of the most sophisticated approachestowards cognitive architectures. Approaches for production machines maybe structured within a cognitive perception-action loop architecture, asshown in FIG. 14, which also defines cognitive technical systems.Cognitive capabilities such as perception, learning, and gainingknowledge allow a machine to interact with an environment autonomouslythrough sensors and actuators. Therefore, in the following, some methodsknown from machine learning that will be suitable for different parts ofa cognitive perception-action loop working in a production system willbe discussed.

If a cognitive technical system simply has a feature representation ofits sensor data input, it may be able to handle a higher volume of data.Moreover, extracting features emphasizes or increases thesignal-to-noise ratio by focusing on the more relevant information of adata set. However, there are many ways of extracting relevant featuresfrom a data set, the theoretical aspects of which are summarized in thefollowing.

In order to select or learn features in a cognitive way, we want to havea method that can be applied completely autonomously, with no need forhuman supervision. One way of achieving this is to use dimensionalityreduction (DR), where a data set X of size t×n is mapped onto a lowerdimension data set Y of size t×p. In this context

^(n) is referred to as observation space and

^(p) as feature space. The idea is to identify or learn a higherdimensional manifold in a specific data set by creating a representationwith a lower dimension.

Methods used to find features in a data set may be subdivided into twogroups, linear and nonlinear, as shown in FIG. 15. Linear dimensionalityreduction techniques seem to be outperformed by nonlinear dimensionalityreduction when the data set has a nonlinear structure. This comes withthe cost that nonlinear techniques generally have longer execution timesthan linear techniques do. Furthermore, in contrast to nonlinear methodslinear techniques allow a straightforward approach of mapping back andforth. The question is whether a linear dimensionality reductiontechnique is sufficient for food processing, or if nonlinear techniquesbring more advantages than costs. The following nonlinear techniques arevery advantageous for artificial data sets: Hessian LLE, LaplacianEigenmaps, Locally Linear Embedding (LLE), Multilayer Autoencoders (ANNAut), Kernel PCA, Multidimensional Scaling (MDS), Isometric FeatureMapping (Isomap), and others.

As a result Isomap proves to be one the best tested algorithms forartificial data sets. We find that the Isomap algorithm seems to be themost applicable nonlinear dimensionality reduction technique for foodprocessing. Therefore Isomap and two linear dimensionality reductiontechniques are introduced below.

Principal Component Analysis

Principal Component Analysis (PCA) enables the discovery of featuresthat separate a data set by variance. It identifies an independent setof features that represents as much variance as possible from a dataset, but are lower in dimension. PCA is known in other disciplines asthe Karhunen-Loève transform and the part referred as Singular ValueDecomposition (SVD) is also a well-known name. It is frequently used instatistical pattern or face recognition. In a nutshell, it computes thedominant eigenvectors and eigenvalues of the covariance of a data set.We want to find a lower-dimensional representation Y with t×p elementsof a high-dimensional data set t×n mean adjusted matrix X, maintainingas much variance as possible and with decorrelated columns in order tocompute a low-dimensional data representation y_(i) for the data setx_(i). Therefore PCA seeks a linear mapping M_(PCA) of size n×p thatmaximizes the term tr(M_(PCA) ^(T) cov(X)M_(PCA)), with M_(PCA)^(T)M_(PCA)=I_(p) and cov(X) as the covariance matrix of X. By solvingthe eigenproblem with

cov(X)M _(PCA) =M _(PCA)Λ,  (Formula 2.3)

we obtain the p ordered principal eigenvalues with the diagonal matrixgiven by Λ=diag(λ₁, . . . , λ_(p)). The desired projection is given by

Y=XM _(PCA)  (Formula 2.4)

gives us the desired projection onto the linear basis M_(PCA). It can beshown that the eigenvectors or principal components (PCs) that representthe variance within the high-dimensional data representation are givenby the p first columns of the matrix M_(PCA) sorted by variance. Thevalue of p is determined by analysis of the residual variance reflectingthe loss of information due to dimensionality reduction.

By finding an orthogonal linear combination of the variables with thelargest variance, PCA reduces the dimension of the data. PCA is a verypowerful tool for analyzing data sets. However, it may not always findthe best lower-dimensional representation, especially if the originaldata set has a nonlinear structure.

Linear Discriminant Analysis

Despite the usefulness of the PCA, the Linear Discriminant Analysis(LDA) may be seen as a supervised dimensionality reduction technique. Itcan be categorized as using a linear method, because it also gives alinear mapping M_(LDA) for a data set X to a lower-dimension matrix Y,as stated for M_(PCA) in equation 2.4. The necessary supervision is adisadvantage if the underlying desire is to create a completelyautonomous system. However, LDA supports an understanding of the natureof the sensor data because it can create features that represent adesired test data set.

Because the details of LDA and Fisher's discriminant are known, thefollowing is a brief simplified overview. Assume we have the zero meandata X. A supervision process provides the class information to divide Xinto C classes with zero mean data X_(c) for class c. We can computethis with

$\begin{matrix}{{S_{w} = {\sum\limits_{c = 1}^{C}{{cov}\left( X_{c} \right)}}},} & \left( {{Formula}\mspace{14mu} 2.5} \right)\end{matrix}$

the within-class scatter S_(w), a measure for the variance of class cdata to its own mean. The between-class scatter S_(b) follows

S _(b)=cov(X)  (Formula 2.6)

Between-class scatter is a measure of the variance of each classrelative to the means of the other classes. We obtain the linear mappingM_(LDA) by optimizing the ratio of the between-class and within-classscatter in the low-dimensional representation using the Fishercriterion,

$\begin{matrix}{{J(M)} = {\frac{M^{T}S_{b}M}{M^{T}S_{w}M}.}} & \left( {{Formula}\mspace{14mu} 2.7} \right)\end{matrix}$

Maximizing the Fisher criterion by solving the eigenproblem for S_(w)⁻¹S_(b) provides C−1 eigenvalues that are non-zero. Therefore, thisprocedure seeks the optimal features to separate the given classes in asubspace with linear projections. LDA thus separates a low-dimensionalrepresentation with a maximized ratio of the variance between theclasses to the variance within the classes.

Isometric Feature Mapping

The PCA and LDA methods produce linear mapping from a high-dimensionaldata set to a low-dimensional representation. This may be expressed aslearning a manifold in an observation space and finding a representationfor this in a lower-dimensional feature space. For data sets with anonlinear structure, such as the artificial Swiss-roll data set, linearprojections will lose the nonlinear character of the original manifold.Linear projections are not able to reduce the dimension in a conciseway: data points in the feature space may appear nearby although theywere not in the observation space. In order to address this problem,nonlinear dimensionality reduction techniques have recently beenproposed relative to the linear techniques. However, it is a prioriunclear whether nonlinear techniques will in fact outperform establishedlinear techniques such as PCA and LDA for data from food processingsensor systems.

Isometric Feature Mapping or the Isomap algorithm attempts to preservethe pairwise geodesic or curvilinear distances between the data pointsin the observation space. In contrast to a Euclidean distance, which isthe ordinary or direct distance between two points that can be measuredwith a ruler or the Pythagorean theorem, the geodesic distance is thedistance between two points measured over the manifold in an observationspace. In other words, we do not take the shortest path, but have to useneighboring data points as hubs to hop in between the data points. Thegeodesic distance of the data points x_(i) in observation space may beestimated by constructing a neighborhood graph N that connects the datapoint with its nearest neighbors in the data set X. A pairwise geodesicdistance matrix may be constructed with the Dijkstra's shortest pathalgorithm. In order to reduce the dimensions and obtain a data set Y,multidimensional scaling (MDS) may be applied to the pairwise geodesicdistance matrix. MDS seeks to retain the pairwise distances between thedata points as much as possible. The first step is applying a stressfunction, such as the raw stress function given by

$\begin{matrix}{{{\Phi (Y)} = {\sum\limits_{ij}\left( {{{{x_{i} - x_{j}}}} - {{{y_{i} - y_{j}}}}} \right)^{2}}},} & \left( {{Formula}\mspace{14mu} 2.8} \right)\end{matrix}$

in order to gain a measure for the quality or the error between thepairwise distances in the feature and observation spaces. Here,∥x_(i)−x_(j)∥ is the Euclidean distance of the data points x_(i) andx_(j) in the observation space with y_(i) and y_(j) being the same forthe feature space. The stress function can be minimized by solving theeigenproblem of the pairwise distance matrix.

The Isomap algorithm thus reduces the dimension by retaining thepairwise geodesic distance between the data points as much as possible.

Classification for Machine Learning

In machine learning, it is not only the extraction of features that isof great scientific interest, but also the necessity of taking decisionsand judging situations. Classification techniques may help a machine todifferentiate between complicated situations, such as those found infood processing. Therefore classifiers use so-called classes thatsegment the existing data. These classes can be learned from a certaintraining data set. In the ongoing research into AI and cognitivemachines, Artificial Neural Networks were developed relatively early inthe process. In comparison, the concepts of Kernel Machines andreinforcement learning appeared only recently but showed increasedcognitive capabilities.

Artificial Neural Networks

Artificial Neural Networks (ANN) have been discussed extensively fordecades. ANN was one of the first successes in the history of ArtificialIntelligence. Using natural brains as models, several artificial neuronsare connected in a network topology in such a way that an ANN can learnto approximate functions such as pattern recognition. The model allows aneuron to activate its output if a certain threshold is reached orexceeded. This may be modeled using a threshold function. Naturalneurons seem to “fire” with a binary threshold. However, it is alsopossible to use a sigmoid function,

$\begin{matrix}{{{f(x)} = \frac{1}{1 + ^{- {vx}}}},} & \left( {{Formula}\mspace{14mu} 2.9} \right)\end{matrix}$

with v as parameter of the transition. For every input connection, anadjustable weight factor w_(i) is defined, which enables the ANN torealize the so-called learning paradigm. A threshold function o can beexpressed using the weight factors W and the outputs from the precedingneurons P, o=W^(T)P, with a matrix-vector notation. The neurons can belayered in a feedforward structure, Multi-Layer Perceptron (MLP) or, forexample, with infinite input response achieved using feedback loops witha delay element in so-called Recurrent Neural Networks. A MLP is afeedforward network with a layered structure; several hidden layers canbe added if necessary to solve nonlinear problems. The MLP can be usedwith continuous threshold functions such as the sigmoid function inorder to support the backpropagation algorithm stated below forsupervised learning. This attempts to minimize the error E in

$\begin{matrix}{{E = {\frac{1}{2}{\sum\limits_{i}\left( {z_{i} - a_{i}} \right)^{2}}}},} & \left( {{Formula}\mspace{14mu} 2.10} \right)\end{matrix}$

from the current output a_(i) of the designated output z_(i), where theparticular weights are adjusted recursively. For an MLP with one hiddenlayer, if h_(j) are hidden layer values, e_(i) are input values, α≧0 isthe learn rate, and ε_(i)=z_(i)−a_(i), then the weights of the hiddenlayer w_(ij) ¹ and the input layer w_(ij) ¹ are adjusted according to,

$\begin{matrix}{{{\Delta \; w_{ij}^{1}} = {{\alpha ɛ}_{i}h_{j}}},} & \left( {{Formula}\mspace{14mu} 2.11} \right) \\{{\Delta \; w_{ij}^{2}} = {\alpha {\sum\limits_{m}{e_{m}w_{mi}^{1}{e_{j}.}}}}} & \left( {{Formula}\mspace{14mu} 2.12} \right)\end{matrix}$

The layers are enumerated starting from the input to the output. Forbackpropagation, the weights are adjusted for the corresponding outputvectors until the overall error cannot be further reduced. Finally, fora classification of C classes, the output layer can consist of either Coutput neurons, representing the probability of the respective class, ora single output neuron that has defined ranges for each class.

ANN can thus learn from or adapt to a training data set and can find alinear or a nonlinear function from N input neurons to C output neurons.This may be used for classification to differentiate a set of classes ina data set.

Kernel Machines

In general, a classification technique should serve the purpose ofdetermining the probability of learned classes occurring based on themeasured data. Classification can be mathematically formulated as a setof classes c_(i)=c₁, . . . , c_(N) in C, with a data set represented byx_(i)∈

^(n), and a probability of p_(i),

p _(i) =p(c _(i) |x _(i))=f _(c)(x _(i),θ)  (Formula 2.13)

The parameter θ may then be chosen separately for every classificationor can be learned from a training data set.

In order to achieve learning, it is desirable to facilitate efficienttraining algorithms and represent complicated nonlinear functions.Kernel machines or Support Vector Machines (SVM) can help with bothgoals. A simple explanation of SVM, or in this particular contextSupport Vector Classification (SVC), is as follows: in order todifferentiate between two classes, good and bad, we need to draw a lineand point out which is which; since an item cannot be both, a binarydecision is necessary, c_(i)∈{−1,1}. If we can only find a nonlinearseparator for the two classes in low-dimensional space, we can find alinear representation for it in a higher-dimensional space, ahyperplane. In other words, if a linear separator is not possible in theactual space, an increase of dimension allows linear separation. Forinstance, we can map with function F a two-dimensional space f₁=x₁,f₂=x₂ with a circular separator to a three-dimensional space f₁=x₁ ²,f_(n)=x₂ ², f_(m)=√{square root over (2)}x₁x₂ using a linear separator,as illustrated in FIG. 16. SVC seeks for this case an optimal linearseparator, a hyperplane,

H={x∈

³ |ox+b=0}  (Formula 2.14)

in the corresponding high-dimensional space for a set of classes c_(i).In three-dimensional space, these can be separated with a hyperplane, H,where o is a normal vector of H, a perpendicular distance to the origin|b|/∥o∥, and o with an Euclidean norm of ∥o∥. In order to find thehyperplane that serves as an optimal linear separator, SVC maximizes themargin given by,

$\begin{matrix}{{{d\left( {o,{x_{i};b}} \right)} = \frac{{{ox}_{i} + b}}{{o}}},} & \left( {{Formula}\mspace{14mu} 2.15} \right)\end{matrix}$

between the hyperplane and the closest data points x_(i). This may beachieved by minimizing the ratio ∥o∥²/2 and solving with the optimalLagrange multiplier parameter α_(i). In order to do this, theexpression,

$\begin{matrix}{{{\sum\limits_{i = 1}^{l}\alpha_{i}} + {\frac{1}{2}{\sum\limits_{j = 1}^{l}{\sum\limits_{k = 1}^{l}{\alpha_{i}\alpha_{j}c_{i}{c_{j}\left( {x_{i} \cdot x_{j}} \right)}}}}}},} & \left( {{Formula}\mspace{14mu} 2.16} \right)\end{matrix}$

has to be maximized under the constraints α_(i)≧0 and Σ_(i)α_(i)c_(i)=0.The optimal linear separator for an unbiased hyperplane is then givenusing,

$\begin{matrix}{{{f(x)} = {{sign}\left( {\sum\limits_{i}{\alpha_{i}{c_{i}\left( {x \cdot x_{i}} \right)}}} \right)}},} & \left( {{Formula}\mspace{14mu} 2.17} \right)\end{matrix}$

allowing a two-class classification.

SVM has two important properties: it is efficient in computationalruntime and can be demonstrated with equations 2.16 and 2.17. First, theso-called support vectors or set of parameters α_(i), associated witheach data point is zero, except for the points closest to the separator.The effective number of parameters defining the hyperplane is usuallymuch less than l, increasing computational performance. Second, the dataenter expression 2.16 only in the form of dot products of pairs ofpoints. This allows the opportunity of applying the so-called kerneltrick with

x _(i) ·x _(j)

F(x _(i))·F(x _(j))=K(x _(i) ,x _(j)),  (Formula 2.18)

which often allows us to compute F(x_(i))·F(x_(i)) without the need ofknowing explicitly F. The kernel function K(x_(i), x_(j)) allowscalculation of the dot product to the pairs of input data in thecorresponding feature space directly. However, the kernel functionapplied throughout the present invention is the Gaussian Radial BasisFunction and has to fulfill certain conditions, as in

K _(G)(x _(i) ,x _(j))=e ^(−γ∥x) ^(i) ^(−x) ^(j) ^(∥) ² ,  (Formula2.19)

with γ as the adjustable kernel parameter.

Because we have so far discussed only binary decisions between twoclasses, we note here that it is also possible to enable soft andmulti-class decisions. The latter can be achieved in steps by a pairwisecoupling of each class against the remaining n−1 classes.

SVC can thus be used to learn complicated data. It structures this datain a set of classes in a timely fashion. Mapping into ahigher-dimensional space and finding the optimal linear separatorenables SVM to use efficient computational techniques such as supportvectors and the kernel trick.

Fuzzy K-Nearest Neighbor

Unlike the previously discussed Support Vector Machines, a lesscomplicated but highly efficient algorithm called the Fuzzy K-NearestNeighbor (KNN) classifier can also separate classes within data. Thealgorithm can categorize unknown data by calculating the distance to aset of nearest neighbors.

Assume we have a set of n labeled samples with membership in a knowngroup of classes. If a new sample x arrives, it is possible to calculatemembership probability p_(i)(x) for a certain class with the vector'sdistance to the members of the existing classes. If the probability ofmembership in class A is 90% compared to class B with 6% and C with just4%, the best results seem to be apparent. In contrast, if theprobability for membership in class A is 45% and 43% for class B, it isno longer obvious. Therefore KNN provides the membership information asa function to the K nearest neighbors and their membership in thepossible classes. This may be summarized with

$\begin{matrix}{{{p_{i}(x)} = \frac{\sum\limits_{j}^{K}{p_{ij}\left( \frac{1}{{{{x - x_{j}}}}^{\frac{2}{m - 1}}} \right)}}{\overset{K}{\sum\limits_{j}}\frac{1}{{{{x - x_{j}}}}^{\frac{2}{m - 1}}}}},} & \left( {{Formula}\mspace{14mu} 2.20} \right)\end{matrix}$

where p_(ij) is the membership probability in the ith class of the jthvector within the labeled sample set. The variable m is a weight for thedistance and its influence in contributing to the calculated membershipvalue.

When applied, we often set m=2 and the number of nearest neighbors K=20.

Reinforcement Learning

In contrast to previous learning methods, which learn functions orprobability models from training data, reinforcement learning (RL) canfacilitate learning using environmental feedback from an agent's ownactions in the long-term, without the need for a teacher. This entailsthe difference between supervised and unsupervised learning. If along-term goal is sought, positive environmental feedback, also known asreward or reinforcement, may support improvement. An agent may learnfrom rewards how to optimize its policy or strategy of interacting withthe real world, the best policy being one that optimizes the expectedtotal reward. RL does not require a complete prior model of theenvironment nor a full reward function. The artificial agents thereforeindicate cognitive capability and act in a similar manner to animals,which may learn from negative results like pain and hunger and frompositive rewards like pleasure and food. In this case we pick that theagent has to use a value function approach, in which it attempts tomaximize its environmental return.

In RL, an agent takes actions, a_(t), in an environment that itperceives to be its current state, s_(t), in order to maximize long-termrewards, r_(t), by learning a certain policy, π. However, before we canstart learning with reinforcement we have to find answers regarding theappropriate agent design. The agent could try to maximize the expectedreturn by estimating the return for a policy π. This agent behavior isalso referred to as value function estimation. The agent may evaluatethe action by estimating the state value using a state-value functionV_(π)(s), considering a certain policy π_(w) that is continuouslydifferentiable, as in

$\begin{matrix}{{V_{\pi}(s)} = {{E\left( {{\sum\limits_{t = 0}^{\infty}{\gamma^{t}r_{t}\text{}s_{0}}} = s} \right)}.}} & \left( {{Formula}\mspace{14mu} 2.21} \right)\end{matrix}$

Using this function the agent may estimate the expected return for agiven state and a following policy. It could also estimate the expectedreturn for an action, following a given state and policy. Therefore, theagent chooses an action considering the given state from thestate-action function or Q-function, as in

$\begin{matrix}{{Q_{\pi}\left( {s,a} \right)} = {{E\left( {{{\sum\limits_{t = 0}^{\infty}{\gamma^{t}r_{t}\text{}s_{0}}} = s},{a_{0} = a}} \right)}.}} & \left( {{Formula}\mspace{14mu} 2.22} \right)\end{matrix}$

The next action therefore relies on a reward function r_(t) and in orderto allow the agent to grant a concession for expected future rewardsover current rewards, the discount factor 0≦γ≦1 may be selected. It ispossible to set how much the agent should discount for future rewards,for instance future rewards are irrelevant for γ=0.

In RL, the methods may be subdivided into groups such as value functionbased methods or direct policy search. Many different actor-criticalgorithms are value function based methods, estimating and optimizingthe expected return for a policy. In order to realize a value functionbased method, the behavior for an artificial agent and the underlyingcontrol problem may be stated as a Markov decision process (MDP). Thesystem perceives its environment over the continuous state set, wheres_(t)∈

^(k) and s₀ as the initial state. It can choose from a possible set ofactions a_(t)∈

^(m) in respect to a stochastic and parameterized policy defined asπ(a_(t)∥s_(t))=p(a_(t)|s_(t), w_(t)), with the policy parameters w∈

^(k). With a learned policy, it can be mapped from states to actionswith respect to the expected rewards r_(t)∈

. The reward after each action relies on r_(t)(s_(t),a_(t)). If noenvironmental model is available, the mentioned actor-critic methods canpotentially develop policy-finding algorithms. The name is derived fromthe theater, where an actor adapts its actions in response to feedbackfrom a critic. This can be obtained using a given evaluation function asa weighted function of a set of features or a so-called basis functionφ(s), which then gives the approximation of the state-value functionwith value function parameters v, as in

V _(π)(s)=φ(s)^(T) v.  (Formula 2.23)

Improving the policy is an optimization issue that may be addressed witha policy gradient. The choice of the policy gradient method is criticalfor convergence and efficiency. Both seem to be met by the NaturalActor-Critic (NAC) algorithm, as described by J. Peters and S. Schaal,“Natural actor-critic”, Neurocomputing, Vol. 71, no 7-9, pp. 1180-1190,2008, where the actor improves using the critic's policy derivative g asin equation 2.24,

g=∇ _(w) log π(a _(t) |s _(t)).  (Formula 2.24)

The steps for improvement of policy parameters of the NAC algorithm arethen calculated using,

w _(t+1) =w _(t) +αĝ,  (Formula 2.25)

where α is the learning rate, and ĝ is the natural gradient calculatedusing the Fisher metric or is derived from the policy as demonstratedwithin the mentioned NAC algorithm publication. The NAC algorithm withLSTD-Q is fully documented at table 1 on page 1183 of J. Peters and S.Schaal, “Natural actor-critic”, Neurocomputing, vol. 71, no. 7-9, pp.1180-1190, 2008. It is applied with a parameterized policy π(a|s)=p(a|s,w) initial parameters w=w_(o) comprising the following steps in pseudocode:

1: START: Draw initial state s₀~p(s_(t)) and select parameters A_(t+1) =0; b_(t+1) = z_(t+1) = 0 2: For t = 0, 1, 2, . . . do 3: Execute: Drawaction a_(t)~π (a_(t) | s_(t)) , observe next state s_(t+1)~p(s_(t+1) |s_(t,) a_(t)), and reward r_(t) = r(s_(t,) a_(t)). 4: Critic Evaluation(LSTD-Q(λ)): Update 4.1: basis functions: {tilde over (φ)}_(t) =[φ(s_(t+1))^(T) ,0^(T)]^(T) , {circumflex over (φ)}_(t) = [φ(s_(t))^(T),∇_(w) logπ(a_(t) | s_(t))^(T)]^(T), 4.2: statistics: z_(t+1) =λz_(t) +{circumflex over (φ)}_(t) ; A_(t+1) = A_(t) + z_(t+1) (φ_(t) − _(γ){tilde over (φ)}_(t) )^(T) ; b_(t+1) = b_(t) + z_(t+1) r_(t,) 4.3:critic parameters: [V_(t+1) ^(T), ĝ_(t+1) ^(T)]^(T) = A_(t+1) ⁻¹b_(t+1,) 5: Actor: If gradient estimate is accurate, update policyparameters 5.1: wt + 1 = wt + α ĝ t + 1 and forget (reset) statistics.END.

The basis functions φ(s) may be represented by mapping the sensor datainput into a feature space as we discussed it elsewhere in thisdocument. In this case the basis functions are equal to the featurevalues. The basis functions may as well be chosen differently or theagent may use raw sensor data. The basis function may as wellincorporate adaptive methods or an own learning step, that maximizeswith the reward function results.

It is important to note that other RL agents are applicable as well.Many other policy learning agent concepts may be applied. It furthermoreis inventive to use other sources as reward signal r_(t) besides theclassification output or quality indicator. For instance it is possibleto apply a post-process or pre-process sensor as reward signal source.The reward function could be the probability value between 0 and 1 or −1to 1 of a measured data of a post-process sensor to be part of a good orbad class, which is determined by a classifier as described above. Incase a pre-process sensor is used for giving a reward r_(t). An RL agentcould find a parameter set to achieve this goal. Thus reinforcementlearning may be a step towards a long-term goal in that it entailslearning a policy from given rewards using policy-finding algorithmssuch as the Natural Actor-Critic.

Cognitive Technical Architecture

An artificial agent is anything that perceives its environment throughsensors and acts in consequence of this through actuators. An agent isdefined as an architecture with a program. The inspirational role modelfor this is natural cognition, and we want to realize a similar actingcognition for technical systems. Therefore, the agent will be equippedwith cognitive capabilities, such as abstracting information, learning,and decision making for a manufacturing workstation. As part of theprocess, this section introduces an architecture that creates andenables agents to manage production tasks. In order to do so, the agentsfollow a cognitive perception-action loop, by reading data from sensorsand defining actions for actuators.

A natural cognitive capability is the capacity to abstract relevantinformation from a greater set of data and to differentiate betweencategories within this information. Transferring this concept fromnatural cognition to the world of mathematical data analysis, acombination of data reduction techniques and classification methods isused according to the present invention to achieve something thatexhibits similar behavior. In industrial production, many manufacturingprocesses can be carried out using a black box model, focusing on theins and outs of the box rather on than what actually happens inside. Theconnections to the black box that may be used in production systems aregenerally sensors and actuators. Sensors such as cameras, microphones,tactile sensors, and others monitor the production processes. Thesesystems also need actuators, such as linear drives or roboticpositioning, in order to interact with its environment. For everyproduction process, these actuators have to be parameterized. In orderto learn how an agent can adaptively control at least one parameter ofthese production systems, many combinations of self-learning algorithms,classification techniques, knowledge repositories, feature extractionmethods, dimensionality reduction techniques, and manifold learningtechniques could be used. The present invention provides also differentcontrolling techniques, both open- and closed-loop, using multipledifferent sensors and actuators. After many simulations and experiments,a simple architecture that demonstrates how these techniques may becombined proved to be successful and reliable, at least for foodprocessing. However, the food processes may be interpreted as a form ofblack box, and may thus be applicable to other types of productionprocesses.

FIG. 17 illustrates a cognitive architecture that may be suitable fordesigning agents that can provide monitoring or adaptive process controlfor production tasks. The diagram describes the unit communication andinformation processing steps. Natural cognition seems to abstractinformation firstly by identifying representative symbolism, such asstructured signals. A similar process can be accomplished usingdimensionality reduction (DR), in which the agent uses a low-dimensionalrepresentation of the incoming sensor data. Natural cognition thenrecognizes whether or not knowledge about the incoming sensationalevents is already present. This step may be achieved by usingclassification techniques that categorize “sensorial” events orcharacteristics. A natural subject may decide to learn or to plan newactions. In order to replicate this, the architecture of the presentinvention offers self-learning techniques that feeds a processing logic.In seeking to achieve quick reactions without the need to start acomplex decisionmaking process, we may also “hard-wire” a sensor inputthat can directly initiate an actuator in using a closed-loop controldesign. Therefore, the architecture of the present invention may bedesigned in respect to four modes of usage, which will be discussedindividually in the following: first, abstracting relevant information;second, receiving feedback from a human expert on how to monitor andcontrol processes, or supervised learning; third, acting on learnedknowledge; and fourth, autonomously controlling processes in previouslyunknown situations. As with other cognitive architectures the aim hereis creating agents with some kind of artificial intelligence orcognitive capabilities related to humans.

The agents may be composed of several components from differentdimensionality reduction and classification techniques, which allow usto compare the performance of composed agents and modules in terms ofoverall food processing quality. Many different dimensionality reductionand classification techniques may be applicable, and some of these havebeen evaluated in the research project. The cognitive architecture ofthe present invention offers the following modules for composing agents:Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA),Isometric Feature Mapping (Isomap), Support Vector Machines (SVM), FuzzyK-Nearest Neighbors (KNN), Artificial Neural Networks (ANN), andreinforcement learning (RL), along with some other methods. Threeembodiments of the present invention of control agents within thisarchitecture would be agent A connecting Isomap, SVM, ANN, and PIDenergy supply control, or agent B connecting Isomap, SVM, and PID energysupply control, or agent C connecting ANN and Fuzzy KNN, for control.

Abstract Relevant Information

In natural human cognition, we abstract or absorb information fromeverything that we hear, feel, and see. Therefore, we only generallyremember the most interesting things. Inspired by this, a technicalcognitive system should similarly abstract relevant information from aproduction process. Working with abstracted features rather than withraw sensor data has certain advantages. Many weak sensor signals may bereduced in dimension to fewer but better signals, resulting in a morereliable feature. Additionally, in order to realize real-time processcontrol, it is necessary to reduce the volume of the incoming sensordata because a greater amount of data may have a significant influencein causing longer execution times for the entire system.

The architecture of the present invention requires a test run in orderto abstract initial information. During this period of agent training,the parameter range of the actuator that will be controlled is altered.In order to determine which information is most relevant, the agentshould explore its own range of actions. After the initial referencetest, the system analyzes the recorded sensor data in order to discoverrepresentative features. The agent may solve feature calculationsseparately for different kinds of sensors, but the sensory units shouldideally be trained to map the sensory input into the learned featurespace. Finding a useful representation of the feature space is criticalbecause the system will only be able to recognize or react to changes inthe feature values. The purpose of the cognitive processing of thepresent invention is to provide as much information as possible for thesubsequent processing steps. However, the raw sensor data containsrepetitions, correlations, and interdependencies that may be neglected.Therefore, in order to abstract the relevant information, the mostsignificant features, or those that contain the most information, shouldbe identified. In order to do this “cognitively”, an agent shouldperform this task without the necessary supervision of a human expert.Therefore, a method of feature extraction is chosen that can be appliedto all of the different kinds of processing tasks and the correspondingsensor data without the need to change parameterization orre-configuration. Manifold learning or dimensionality reductiontechniques satisfy this need. They can reduce a sensor data set X ofdimension n in observation space to a data set of dimension p in featurespace. Often, the new quantity p is much less than n. However, manylinear and nonlinear dimensionality reduction techniques have been triedand tested for different purposes. The present invention provides asuitable feature extraction technique for production workstations,complying with the following requirements the feature extraction methodworks transparently and is able to display the processing steps to theuser. The feature extraction method is able to run unsupervised. Thefeature extraction method is executable within a reasonable time-framefor configuration, especially during processing. The extracted featurescontain enough process information for reliable classification withinseveral food loads.

In essence, PCA seeks orthogonal linear combinations that represent agreater data set. These may be calculated for incoming sensor datavectors. These eigenvectors may serve as features for classification upto a threshold d. Feature extraction combined with classification may beachieved using Linear Discriminant Analysis. Analyzing the same data setusing LDA and three learned quality classes defined as “good”, “medium”,and “bad” provides another set of features. Feature extraction may alsobe achieved using the Isomap algorithm. Unfortunately, the nonlinearfeature cannot be displayed in the same way as the linear featureextraction of LDA and PCA. The extracted features of the methods namedabove are compared in the following. The LDA feature seems to containmore details than any one of the PCA features. Using this method ofcalculating, the LDA features seem to contain more process informationin fewer features than PCA because they are especially designed toseparate the desired classes. Furthermore, it is possible to display thecalculated features using PCA and LDA in a way that makes these twomethods more transparent than Isomap. The user gets an idea of what aprocess looked like if a feature is identified in a process video simplyby looking at it. PCA and Isomap have the advantage that they can rununsupervised, which is not possible with LDA. Therefore, LDA merelyserves as a comparison to PCA, but is not considered as an alternativefor the desired architecture. Furthermore, the LDA feature seems to bevery individualized for a particular process. Isomap has considerablyhigher execution times for analysis and out-ofsample extension.Therefore, if classification with PCA achieves sufficient results, thenit is more applicable to the system under research. Therefore, themethod of choice would be PCA, unless Isomap shows a significantlybetter performance toward the first object of the present invention. Wehave to postpone the final choice of dimensionality reduction techniquesbecause the most important quality measures are the experimentalresults, which are the basis of the present invention.

In essence, dimensionality reduction may allow agents to abstractrelevant information in terms of detecting variances and similaritiesduring a training trial. This helps the agent to process only a fewfeature values compared to the significantly higher volume of raw sensordata. Furthermore, dimensionality reduction may support the perceptionof similarities in unknown situations, for instance similar foodprocessing characteristics such as food size and form, even if thesehave not been part of the training. This may improve the adaptability ofthe agents to unknown but similar situations.

Supervised Learning from Human Experts

In natural human cognition, for instance in childhood, we often learnfrom others how to manage complex tasks. Similarly, a machine shouldhave the possibility of learning its task initially from a human expert.Supervised learning seems to be the most efficient way of setting up acognitive agent for production. In industrial production, a qualifiedhuman supervisor is usually present when the production system is beinginstalled or configured. The architecture that we are examining useshuman-machine communication in order to receive feedback from an expert,for instance through an intuitive graphical user interface on atouch-screen tablet computer. As mentioned above, at least one testaction per actuator or test run is needed in this architecture as aninitial learning phase. During these tests, the agent executes oneactuator from within the desired range of actions, and the sensor datainput is stored. After this run, an expert provides feedback concerningwhether the robot has executed the actuator correctly, or if the actionwas unsuccessful or undesirable. The feedback may come in many differentcategories so that different kinds of failures and exit strategies maybe defined. A classification technique may then collect the featurestogether with the corresponding supervisory feedback. Combined withlookup tables, the classifier module will serve as knowledge and as aplanning repository for a classification of the current system state.How an agent may perform its own actions and give itself feedback willbe of importance for the next section; this section mainly covers thecognitive capability of learning from a human expert, and theapplication of this knowledge for monitoring purposes.

Support Vector Machines, Fuzzy K-Nearest Neighbor, and Artificial NeuralNetworks as classification techniques have been discussed. The more thatthe human expert teaches the machine, the likelier it is that the systemwill achieve the desired goal. In order to save costs, the necessaryhuman supervisor time should be minimized to just one or two referencetests, if possible.

Semi-Supervised Learning

The previous discussion shows how agents in the investigated cognitivearchitecture perceive their surroundings and learn from a human expert,as well as displaying their knowledge in terms of monitoring. Theprovided monitoring signal based on selected features is obtained fromdifferent sensors that are interpreted using a trained classifier. Thismonitoring signal seems to have improved quality and may be applicableto the control of process parameters. The agent would then change itsposition from observing the processing to actually acting upon thegained knowledge. However, if an agent is also applicable to processcontrol in industrial processing, it has to fulfill many requirementswith a performance close to perfection. The following are some of therequirements for the underlying cognitive architecture: The processcontrol module should be capable of completing at least onecontrol-cycle from sensor input to actuator output. The controlledparameter should have an effect on the process outcome when altered,while simultaneously responding in a timely fashion. The process controlmodule should be optimized in terms of providing a balance of reliablestability and necessary dynamics.

In order to realize a robust process control that is suitable forindustrial production processes, a fast or real-time closed-loop controlis often required. The advantage of the architecture under investigationis that the use of features rather than raw sensor data permits fastercompletion of control-loops with a minimal loss of information. In thisarchitecture, any kind of controller design may be implemented that fitswith the classification output. A simple version would have threepossible classification output values: under baked, class I; correct,class II; and over baked, class III. This may be expressed using

$\begin{matrix}{{y_{e} = {\left\lbrack {- 101} \right\rbrack \left\lfloor \begin{matrix}p_{I} \\p_{II} \\p_{III}\end{matrix} \right\rfloor}},} & \left( {{Formula}\mspace{14mu} 3.1} \right)\end{matrix}$

where p are the class probabilities and y_(e) the quality indicator.

A PID controller could adjust a parameter of the system's actuatorsaccording to the monitoring signal discussed above concerning supervisedlearning from human experts. Combining PID-control with theclassification results enables the agents to perform energy suppliedcontrolled processing. This can be realized as shown in

$\begin{matrix}{{c_{t} = {{Pe}_{t} + {I{\sum\limits_{i = {t - n}}^{t - 1}e_{i}}} + {D\left( {e_{t} - e_{t - 1}} \right)}}},} & \left( {{Formula}\mspace{14mu} 3.2} \right)\end{matrix}$

with P for proportional, I for integral, and D for derivative behavior.The goal is to minimize the error e_(t) between the quality indicatorthe output of the classification module, and the desired value of 0.0.In this context, the inventive applicability of the desired value independency of a probability class related quality indicator gives theopportunity to vary this value to optimize the desired process results.One approach describes a PID control with an ANN and correspondingexperiments. Another investigates the usage of an SVM classificationmodule to control food processing.

Unsupervised Learning

As suggested, a self-learning mechanism is integrated into the system ofthe present invention. A novelty check on the basis of the trainedfeatures can detect new or previously unknown situations. In thesecases, the system performs another test action and classifies the newfood using the previously trained features. This time, it does not needto consult a human expert; it can map the gained knowledge onto the newfood autonomously and can adjust the process control appropriately.

In order to achieve process feedback control, the monitoring signaly_(e) is used as the control variable. As actuating variable, whichcould possibly be any alterable process parameter with interrelationshipto y_(e), the energy supply seems suitable for its low inertia and itsstrong relation to y_(e). Its magnitude is calculated by the PIDalgorithm as shown in equation 3.2. In order to achieve process control,the agent closes the loop by connecting the monitoring signal to a PIDcontroller, as is shown in equation 3.2. The feedback controller isdesigned as a single-input-single-output (SISO) control system, whichreceives the monitoring signal y_(e) from the classification unit, with0<y_(e)≦1 for too low and −≦y_(e)<0 for too high energy supply, and usesthis as reference value to minimize controller error.

The previous description outlined how the cognitive agents learned fromhuman expert feedback. It should be possible for the cognitive system tolearn from its own actions, or to give itself feedback. This kind ofcognitive capability may be attained with reinforcement learning (RL). Aclassifier may take over the role of giving feedback and provide a RLagent with rewards for its own actions. The agent then learns a policyon how to act or how to bake based on the feedback or on rewardsreceived for its previous performance. In order to test this, thelearning task is therefore for the agent to learn how to process food onthe basis of gained knowledge at different velocities without furtherhuman expert supervision.

In order to achieve the given learning task using reinforcementlearning, a reliable reward function is needed. As the system hasmultiple sensor data inputs, a classifier identifying features of a goodbaking, such as a Support Vector Machine, may serve as reward functionr_(t), as is shown in FIG. 23. These rewards may fulfill the role of acritic in the Natural Actor-Critic method, which is described before.Therefore, the next action that the agent chooses is absolute energysupply, a_(t). The chosen action depends on the learned policy, as isshown in

π(a _(t) |s _(t))=p(a _(t) |s _(t) ,w _(t)).  (Formula 4.1)

The policy parameters w_(t) relies on the gradient ĝ and w_(t−1), as inequation 2.25. However, for a full review of the applied algorithmplease consult the Natural Actor-Critic Algorithm with least-squarestemporal difference learning, LSTD-Q(λ). The policy should enable theagent to map from states, s_(t), to actions, a_(t), by learning fromrewards, r_(t). The rewards naturally influence the policy parameters.The best policy of the RL agent of the present invention underinvestigation has been found with a sigma function,

$\begin{matrix}{{{\pi \left( {\varphi \left( {a_{t}s_{t}} \right)} \right)} = \left. {{L_{m}\frac{1}{1 + ^{{- w_{t}^{T}}{\varphi {(s_{t})}}}}} + \eta}\Rightarrow a_{t + 1} \right.},} & \left( {{Formula}\mspace{14mu} 4.2} \right)\end{matrix}$

where L_(m) is the maximum allowed power and η is the exploration noisedetermined by the product of a random number from −1 to 1 and theexploration parameter ε. The present invention has investigated modulesthat are suitable for a cognitive architecture for food productionmachines within a cognitive perception-action loop connecting sensorsand actuators. Cognitive capabilities are: to abstract relevantinformation; to learn from a human expert; to use the gained knowledgeto make decisions; and to learn how to handle situations that the agenthas not previously been trained in.

As already mentioned above the previously discussed machine learningtechniques may be implemented in any herein described embodiment of aheat treatment monitoring system.

In the following, an embodiment of a heat treatment monitoring system100 illustrated in FIGS. 18A and 18B will be described. The heattreatment monitoring system comprises an oven 100 and a monitoringapparatus 150 as described above with regard to FIGS. 1A and 1B. Theembodiment as described with regard to FIG. 18A should, however, not berestricted to the usage of the window 130 as described above, thus anykind of window 1800 adapted to permit the camera 160 to observe the foodto be heated may be used. The embodiment of the monitoring apparatus 150should further not be restricted to the employment within the embodimentof FIGS. 1A and 1B, but may be further employed within baking orpre-baking lines or food heating lines as described with regard to FIGS.8 to 10 or in any other embodiment as described above.

A block diagram of an embodiment of the monitoring apparatus 150 isshown in FIG. 18B. The monitoring apparatus 150 and the monitoringsystem 100, accordingly, comprises a sensor unit 1810 having at leastone sensor 1815 to determine current sensor data of food being heated, aprocessing unit 1820 to determine current feature data from the currentsensor data, and a monitoring unit 1830 adapted to determine a currentheating process state in a current heating process of monitored food bycomparing the current feature data with reference feature data of areference heating process. The heat treatment monitoring system furthercomprises a learning unit 1840 adapted to determine a mapping of currentsensor data to current feature data, and to determine reference featuredata of a reference heating process based on feature data of at leastone training heating process. The monitoring apparatus 150 furthercomprises a classification unit 1850 adapted to classify the type offood to be heated and to choose a reference heating processcorresponding to the determined type of food. It should be emphasizedthat the respective units 1820, 1830, 1840, and 1850 may be providedseparately or may also be implemented as software being executed by aCPU of the monitoring apparatus 150.

The sensor unit 1810 comprises at least one sensor 1812, wherein asensor 1812 may be any sensor as described in the description above, inparticular a camera 160 as described with respect to FIGS. 1A and 1B,any sensor of the sensor system 850 described with respect to FIG. 7 or8 or the sensor system described with regard to FIG. 12. In particular,the at least one sensor 1812 of the sensor unit 1810 comprises at leastone of hygrometer, insertion temperature sensor, treatment chambertemperature sensor, acoustic sensors, scales, timer, camera, imagesensor, array of photodiodes, a gas analyser of the gas inside thetreatment chamber, means for determining temperature profiles ofinsertion temperature sensors, means for determining electromagnetic oracoustic process emissions of the food to be treated like light or soundbeing reflected or emitted in response to light or sound emitters orsources, means for determining results from 3D measurements of the foodto be heated including 3D or stereo camera systems or radar, or meansfor determining the type or constitution or pattern or opticalcharacteristics or volume or the mass of the food to be treated.According to this embodiment it is beneficial to use as much sensor dataas input as feasible. Which sensor signal provides the best informationis hard to predict. As the algorithms detect the variance of a referencebake, the learning unit 1840 used to implement machine learning maychoose different sensor data for individually different baking products.Sometimes, volume and color variance may be the most significant data,sometimes it may be humidity, temperature and weight.

In an embodiment, the sensor unit 1810 comprises the camera 160 as theonly sensor 1812, which leads to the advantage that no further sensorhas to be integrated in the monitoring apparatus 150. Thus, themonitoring apparatus 150 may be formed as a single and compact casingbeing mounted to an oven door of the oven 110. It is, however, alsopossible to provide a sensor data input interface 1814 at the monitoringapparatus 150, by which current sensor data of the above mentionedsensors can be read by the sensor unit 1810 and transferred to theprocessing unit 1820. The current sensor data of the sensors 1812 arenot necessarily raw data but can be pre-processed, like HDRpre-processed pixel data of the camera 160 or pre-processed sensor dataof the laser triangulation sensors, which may contain, e.g. a calculatedvalue of volume of the observed food piece.

The processing unit 1820, the monitoring unit 1830, the learning unit1840 and the classification unit 1850 cooperate to provide a user withan optimized food heating result based on machine learning techniques asdescribed above.

Herein, the processing unit 1820 and the learning unit 1840 are providedto reduce the amount of current sensor data of the above at least onesensor 1812. In particular, the learning unit 1840 is adapted todetermine a mapping of current sensor data to current feature data bymeans of a variance analysis of at least one training heating process,to reduce the dimensionality of the current sensor data.

The learning unit 1840 may be integrated in the monitoring apparatus 150or may be an external unit located at another place, wherein a dataconnection may be provided, e.g. via Internet (as described below withregard to the usage of PCA-loops). The at least one training heatingprocess may thus be based on current sensor data of the sensor unit 1810of the local monitoring apparatus 150, but also be based on currentsensor data of sensor units of further monitoring apparatuses atdifferent places (on the world), provided the case the type of sensordata is comparable with each other. By means of training heatingprocesses, the sensor data are reduced in dimensionality, wherein sensordata with the highest variance over time is weighted most.

The variance analysis performed by the learning unit 1840 comprises atleast one of principal component analysis (PCA), isometric featuremapping (ISOMAP) or linear Discriminant analysis (LDA), or adimensionality reduction technique, which have been described in alldetail above.

An interpretation and selection of dominant features may thus beperformed by applying PCA or principle component analysis to a sequenceof food processing data. As described above in this way the features maybe sorted by variance and the most prominent may be very beneficial formonitoring. By performing the analysis as described above, a mapping canbe derived for mapping sensor data to feature data being reduced indimensionality and being characteristic for the heating process beingperformed and being monitored by the monitoring apparatus 150. Themapping, which may be also received from an external server, or may bestored in a memory in the monitoring apparatus 150, is then applied bythe processing unit 1820 to map the incoming current sensor data fromthe sensor unit 1810 to current feature data, which are then transmittedto the monitoring unit 1830. It is emphasized that in some cases, the“mapping” might be for some sensor data an identify mapping, thus someof the sensor data might be equal to the respective feature data, inparticular with regard to pre-processed sensor data already containingcharacteristic values like the absolute temperature within the heatingchamber, a volume value of the food to be heated, a humidity value ofthe humidity within the heating chamber. However, the mapping ispreferably a mapping, in which the dimensionality of the data isreduced. The learning unit may be further adapted to determine a mappingof current feature data to feature data by means of a variance analysisof at least one training heating process to reduce the dimensionality ofthe current feature data.

The monitoring unit 1830 is then adapted to determine a current heatingprocess state in a current heating process of monitored food bycomparing the current feature data with reference feature data of areference heating process.

During monitoring, one of the desired interests is to interpret thecurrent feature data and arrive with a decision about regular andirregular processing. With the named method it is possible to collectfeatures of regular behaviour and then assume irregular behaviour, oncefeature values differ from the previously learned regular behaviour.This may be supported by including classifiers such as Support VectorMachines or k-nearest neighbours as described above. The monitoring unit1830 may be adapted to determine at least one action of at least oneactuator based on the determined current feature data or current heatingprocess state, wherein the control unit 1300 as described above may beimplemented in the monitoring unit 1830. Thus, the monitoring unit 1830may be adapted to execute all machine learning techniques as describedabove.

According to an embodiment, the reference feature data of a referenceheating process is compared with current feature data to determine acurrent heating process state. The reference feature data may bepredetermined data received from an external server or stored in amemory of the monitoring apparatus 150. In another embodiment, thelearning unit 1840 (external or internal of the monitoring apparatus150) may be adapted to determine reference feature data of a referenceheating process by combining predetermined feature data of a heatingprogram with a training set of feature data of at least one trainingheating process being classified as being part of the training set by anuser. The heating program can be understood as a time dependent sequenceof feature data being characteristic for a certain kind or type of foodto be heated.

For example, a reference heating process or a predetermined heatingprogram may be a sequence of feature data in time of a certain kind offood to be heated like a Croissant, which leads to an optimized heatingor baking result. In other words, if the current feature data exactlyfollows the time dependent path of the reference feature data points inthe feature space having the dimensionality of the number of chosenrelevant features, the food will be heated in an optimized way after apredetermined optimized time, i.e. the Croissant will be bakenperfectly. The optimized time may be dependent on the temperature withinthe heating or baking chamber.

Combining predetermined feature data of a heating program with atraining set of feature data of at least one training heating processbeing classified as being part of the training set by an user means thata point cloud of feature data in the feature space of the training set(i.e. of at least one training heating process being considered as being“good” by a user) is averaged for each time point (a center point of thepoint cloud is determined within the feature space) and then used toadapt the predetermined heating program. This can be done by furtheraveraging the features of the heating program and the features of thetraining set equally or in a weighted way for each time point. Forexample, the weighting of the training set may be 25%, the weighting forthe predetermined heating program may be 75%.

Thus, at least one reference bake (training heating process) may betaken to optimite subsequent bakes. Further feedback from subsequentbakes may optimize the individual baking programs accordingly.Accordingly, it is possible to achieve more consistent baking quality,if the current bake is being adapted by the current sensor data and itscalculated alterations taken from the difference of the current bake andthe so called “ground truth” (reference heating process), which is thebaking program (predetermined heating program) combined with the featuredata of at least one reference bake (training set) as well as thefeature data from later feedback (training set) to the baking programand its according sensor data.

Thus, it is possible to calculate significant features withcorresponding feature values from the sensor data of a reference bakecombined with the time elapsed of the baking program. Here, it isfeasible to use many different feature calculation variations and thensort them by variance. A possible mechanism to sort by variance isPrinciple Component Analysis (PCA) described above. When severalfeatures and feature values over time are calculated from a referencebake it is feasible to sort these sets of features and feature valuesover time with the PCA.

It is possible to automatically design a control algorithm for therepeating bakes by taking at least one of the most significant featuresand feature value data sets, preferably the one with most significantvariance. If several reference bakes are present it is preferable totake the one with highest variance and highest feature value repetition.

To implement the above possibility to adapt the predetermined heatingprogram to form a “ground truth”, i.e. the reference heating process,the monitoring apparatus 150 may further comprise a recording unit 1822to record current feature data of a current heating process, wherein thelearning unit 1840 is adapted to receive the recorded feature data fromthe recording unit 1822 to be used as feature data of a training heatingprocess.

The classification unit 1850 may be provided to classify the type offood to be heated. This may be done by image processing of an pixelimage of the food to be heated, e.g. by face recognition techniques.After determining the type of food to be heated (bread roll, muffin,croissant or bread), the classification can be used to select arespective predetermined heating program or stored reference heatingprocess corresponding to the respective type of food to be heated. Inaddition, sub-categories can be provided, for example small croissant,medium croissant, or big size croissant. Different reference heatingprocesses may also stored with regard to non food type categories. Forexample, there may be a reference heating program corresponding todifferent time dependent environments or oven parameters.

For example, weather data may be implemented in the baking procedure ofthe present invention. By means of the known geographic altitude of thegeometric position of the baking oven, the boiling point may bedetermined, thus leading to an adaptation of the baking program.Moreover, local pressure, temperature, and humidity data of theenvironment of an oven may be used to further adapt the baking program.Thus, these data might be recorded and used as index data for certainreference heating programs, which then can be looked up in the memory.

Further, statistics of loads, units and corrections may also be used asdata for the inventive self-learning baking procedure. Thus a bakingdata history may help to improve the baking procedure of the presentinvention. By means of the distributed feedback being accounted for byrole definition, the baking process of the present invention may beimproved. The heat treatment monitoring systems in use may be furtherdisplayed on a zoomable world map.

Moreover, the baking data history may also take into account the amountof baking products produced over time. The heat treatment monitoringsystem may search the baking data history for periodically occurringminima and maxima of the production and estimate the occurrence of thenext minimum or maximum. The heat treatment monitoring system may theninform a user of the system whether too many or too little food isproduced for the time period of the expected minimum or maximum.

The current heating process state is determined by comparing the currentfeature data with reference feature data. The comparing may be thedetermination of the distances of the current feature data and thereference feature data for each time point of the reference heatingprogram. Thus, by determining the nearest distance of the determineddistances, the time point of the nearest distance can be looked up inthe reference heating program and thus, for example, a remaining bakingtime can be determined.

As described above, the sensor unit 1810 may comprise a camera like thecamera 160 recording a pixel image of food being heated, wherein thecurrent sensor data of the camera corresponds to the current pixel dataof a current pixel image.

Feature detection for image processing may comprise the following steps:detection of edges, corners, blobs, regions of interest, interestpoints, processing of color or grey-level images, shapes, ridges, blobsor regions of interest or interest points. Feature from sensor data mayalso comprise target amplitude selection or frequency-based featureselection.

Herein, edges are points where there is a boundary (or an edge) betweentwo image regions. In general, an edge can be of almost arbitrary shape,and may include junctions. In practice, edges are usually defined assets of points in the image which have a strong gradient magnitude.Furthermore, some common algorithms will then chain high gradient pointstogether to form a more complete description of an edge. Thesealgorithms usually place some constraints on the properties of an edge,such as shape, smoothness, and gradient value. Locally, edges have a onedimensional structure.

The terms corners and interest points are used somewhat interchangeablyand refer to point-like features in an image, which have a local twodimensional structure. The name “Corner” arose since early algorithmsfirst performed edge detection, and then analysed the edges to findrapid changes in direction (corners). These algorithms were thendeveloped so that explicit edge detection was no longer required, forinstance by looking for high levels of curvature in the image gradient.It was then noticed that the so-called corners were also being detectedon parts of the image which were not corners in the traditional sense(for instance a small bright spot on a dark background may be detected).These points are frequently known as interest points, but the term“corner” is used by tradition.

Blobs provide a complementary description of image structures in termsof regions, as opposed to corners that are more point-like.Nevertheless, blob descriptors often contain a preferred point (a localmaximum of an operator response or a center of gravity) which means thatmany blob detectors may also be regarded as interest point operators.Blob detectors can detect areas in an image which are too smooth to bedetected by a corner detector. Consider shrinking an image and thenperforming corner detection. The detector will respond to points whichare sharp in the shrunk image, but may be smooth in the original image.It is at this point that the difference between a corner detector and ablob detector becomes somewhat vague. To a large extent, thisdistinction can be remedied by including an appropriate notion of scale.Nevertheless, due to their response properties to different types ofimage structures at different scales, the LoG and DoH blob detectors arealso mentioned in the article on corner detection.

For elongated objects, the notion of ridges is a natural tool. A ridgedescriptor computed from a grey-level image can be seen as ageneralization of a medial axis. From a practical viewpoint, a ridge canbe thought of as a one-dimensional curve that represents an axis ofsymmetry, and in addition has an attribute of local ridge widthassociated with each ridge point. Unfortunately, however, it isalgorithmically harder to extract ridge features from general classes ofgrey-level images than edge-, corner- or blob features. Nevertheless,ridge descriptors are frequently used for road extraction in aerialimages and for extracting blood vessels in medical images.

The current pixel data may comprise first pixel data corresponding to afirst color, second pixel data corresponding to a second color, andthird pixel data corresponding to a third color, wherein the first,second and third color corresponds to R, G and B, respectively. Herein,an illumination source for illuminating the food with white light isadvantageous. It is, however, also possible to provide a monochromaticillumination source in a preferred wavelength area in the opticalregion, for example at 600 nm, to observe a grey pixel image in therespective wavelength.

Due to the provision of separate analysis of R, G and B pixel values, itis possible to implement an algorithm which may learn bread colors.Here, it is essential to segment the bread pixels from the oven pixels,which may be done by color. It is of advantage to use high dynamic range(HDR) pre-processed pictures to have more intensity information to havethe best segmentation. Thus, the camera is preferably adapted togenerate HDR processed pixel images as current pixel data. Herein, alsologarithmic scaling may be implemented, wherein the camera is adapted torecord a linear logarithmic or combined linear and logarithmic pixelimages. To learn the bread pixels an Artificial Neural Network with backpropagation or an SVM class as described above may be used, which aretrained with pictures, where the oven is masked manually.

As an example, it may be that for baking rolls the most significantvariance during the bake is a change in color (intensity change ofpixels) and a change in volume (change in number of pixels with certainintensity). This may be the two most significant features during thereference bake or reference heating process and the correspondingfeature values change over time. This creates a characteristic of thebaking process. For instance the feature value representing the volumechange may have a maximum after 10 minutes of 20 minutes and the colorchange after 15 minutes of 20 minutes of a bake. It is then possible todetect in repeating bakes by means of a classifier such as theaforementioned Support Vector Machine in the incoming sensor data of therepeating bake that the highest probabilities match in the referencebake or reference heating program. It may be that for instance the colorchange in the repeated bake has a maximum after 5 minutes for the volumechange. The time difference of the repeating bake and the reference bakethus would be 50%. This would result in an adaptation of the remainingbake time by at least 50%. Here, an elapsing time of 5 minutes insteadof 15.

Further, it may be possible to integrate an impact factor that mayinfluence the impact of the control algorithm to the repeating bakingprogram. This may be either automatically, such that the number ofreference bakes influences the confidence factor, or such that it ismanually set to a certain factor. This may as well be optimized by meansof a remote system using information technology described earlier.

Moreover, it may be especially possible to change the temperature withinthis system by a change of a feature representing the color change. Asit is described it is possible to calculate features representing thecolor change (change of intensity of pixels). It is feasible tonormalize the pixel intensity. After normalization it is possible toadjust the temperature according to the change of color. If for exampleafter 75% of remaining time there has not been the expected change incolor the temperature may be risen, or if there has been more colorchange than expected from the reference bake the temperature may belowered.

The monitoring apparatus 150 may further comprise a control unit 1860adapted to change a heating process from a cooking process to a bakingprocess based on a comparison of the current heating process statedetermined by the monitoring unit with a predetermined heating processstate. The current heating process state is calculated as above bydetermining the time point of “nearest distance”.

By comparing the time points of the predetermined heating process stateand the calculated time point, the heating process is changed, if thecalculated time point is later then the time point of the predeterminedheating process state. For example, as a rule of dumb, a proofing shallbe finished after a volume change of 100% of the food to be heated,thus, if the bread roll or the Croissant has twice a volume, theproofing shall stop and the baking procedure shall start. The volumechange of the bread or food to be baked may be detected by the camerapixel features in a very efficient way. The heat treatment machine to becontrolled may be an integrated proofing/baking machine, however, alsodifferent machines for proofing or baking may also be controlled.

To simplify the calculations and to ensure repeatable results, it ispreferred if the heating temperature is kept constant in a currentheating process.

The control unit 1860 is further adapted to stop the heating processbased on a comparison of the current heating process state determined bythe monitoring unit with a predetermined heating process statecorresponding to an end point of heating. The control unit 1860 may beadapted to alert a user, when the heating process has to be ended.Therefore, the monitoring apparatus may comprise an alert unit 1870 anda display unit 1880. The display unit 1880 is provided to indicate thecurrent heating process state, for example the remaining heating orbaking time. The display unit 1880 may further show a current pixelimage of the inside of the heat treatment chamber for visual monitoringof the food to be heated by a user. The control unit 1860 may be adaptedto control the display unit 1880 being adapted to indicate a remainingtime of the heating process based on a comparison of the current heatingprocess state determined by the monitoring unit with a predeterminedheating process state corresponding to an end point of heating and/or todisplay images of the inside of the heat treatment chamber.

The control unit 1860 is further connected to an output interface 1890for controlling actuators as described above or below like a temperaturecontrol of a heating chamber, means to adapt humidity in the heattreatment chamber by adding water, or a control of the ventilatingmechanism (ventilating shutter). The actuators may further include meansfor adapting the fan speed, means for adapting the differential pressurebetween the heat treatment chamber and the respective environment, meansfor setting a time dependent temperature curve within the heat treatmentchamber, means for performing and adapting different heat treatmentprocedures like proofing or baking, means for adapting internal gas flowprofiles within the heat treatment chamber, means for adaptingelectromagnetic and sound emission intensity of respectiveelectromagnetic or sound emitters for probing or observing properties ofthe food to be heated.

In particular, the control unit 1860 is adapted to control a temperaturecontrol of a heating chamber, means to adapt humidity in the heattreatment chamber by adding water or steam, a control of the ventilatingmechanism, means for adapting the fan speed, means for adapting thedifferential pressure between the heat treatment chamber and therespective environment, means for setting a time dependent temperaturecurve within the heat treatment chamber, means for performing andadapting different heat treatment procedures like proofing or baking,means for adapting internal gas flow profiles within the heat treatmentchamber, means for adapting electromagnetic and sound emission intensityof respective electromagnetic or sound emitters for probing or observingproperties of the food to be heated.

A heat treatment monitoring method of the present invention comprisesdetermining current sensor data of food being heated; determiningcurrent feature data from the current sensor data; and determining acurrent heating process state in a current heating process of monitoredfood by comparing the current feature data with reference feature dataof a reference heating process. The method preferably further comprisesdetermining a mapping of current sensor data to current feature dataand/or to determine reference feature data of a reference heatingprocess based on feature data of at least one training heating process.In addition, the method comprises determining a mapping of currentsensor data to current feature data by means of a variance analysis ofat least one training heating process to reduce the dimensionality ofthe current sensor data. The method further comprises determining amapping of current feature data to feature data by means of a varianceanalysis of at least one training heating process to reduce thedimensionality of the current feature data. The variance analysispreferably comprises at least one of principal component analysis (PCA),isometric feature mapping (ISOMAP) or linear Discriminant analysis(LDA), or a dimensionality reduction technique. The method furthercomprises preferably determining reference feature data of a referenceheating process by combining predetermined feature data of a heatingprogram with a training set of feature data of at least one trainingheating process being classified as being part of the training set by anuser. In addition, by the method of the present invention, currentfeature data of a current heating process may be recorded, wherein therecorded feature data is used as feature data of a training heatingprocess.

Furthermore, the method may comprise classifying the type of food to beheated and to choose a reference heating process corresponding to thedetermined type of food. Preferably, a heating process is changed from aproofing process to a baking process based on a comparison of thecurrent heating process state with a predetermined heating processstate. The heating temperature is preferably kept constant in a currentheating process. Preferably, the heating process is stopped based on acomparison of the current heating process state determined by themonitoring unit with a predetermined heating process state correspondingto an end point of heating. In an advantageous embodiment, a user isalerted, when the heating process has to be ended.

According to another embodiment of the monitoring apparatus 150, machinelearning may be used for a multi input and multi output (MIMO) system.In particular, an adjusting system for added water, remaining bakingtime and/or temperature may be implemented by a heat treatmentmonitoring system using machine learning techniques.

The system is collecting all sensor data during the reference bake. Incase of humidity, at least one hygrometer detects a reference value forthe humidity over bake time during the reference bake. When repeating abaking of the same product the amount of water to be added may bedifferent. The amount of baked products may be different, the oveninside volume may be different, or there may be more or less ice orwater on the baked products when loading the oven.

Next to other adaptations, the control system according to the inventionadds as much water as needed to achieve similar conditions compared tothe reference baking. As the remaining bake time may be adapted by thecontrol system, the time at which the water will be added changes aswell. Instead of using a fixed time, such as to add 1 liter of waterafter 10 minutes of a 20 minutes baking program, according to thisembodiment the system will add as much water as needed to hit thereference bake humidity level after 50% of elapsed time.

Once irregular behaviour is recognized in an implementation of thisinvention, this signal or irregularity and it's corresponding amplitudemay be used to adjust processing devices such as mixers (energy inducedinto dough), dough dividers (cutting frequency), or industrials ovens(baking program times or temperature) within a food production process.

According to another embodiment the observation of the food within thebaking chamber may be done “live”, thus a live view of the oven insideenables a remote access of the baking process. Also remote ovenadjustment may be possible to improve the baking behavior of aself-learning heat treatment monitoring system.

In an embodiment “perception”, “cognition”, and “action” (P-C-A) loops,cognitive agents, and machine learning techniques suitable forindustrial processes with actuators and intelligent sensors may be used.Transferring cognitive capabilities, knowledge, and skills, as well ascreating many interacting P-C-A loops will be advantageous in acognitive factory.

Only very few food production processes are unique. The majority of foodproduction processes run at different facilities or at different timesperforming identical tasks in similar environments. Still, often no orlimited information exchange exists between these processes. The samefood processing stations often require an individual configuration ofevery entity managing similar process tasks. In order to increase thecapability of machines to help each other it is advantageous to combinein space or time distributed P-C-A loops. Certain topics arise toapproach this aim: In order to enable skill transfer between differententities it is advantageous to establish a reliable and adaptableMulti-P-C-A-loop topology. This metasystem should be able to identifysimilar processes, translate sensor data, aquire features, and analyzeresults of the different entities. Dimensionality reduction, clustering,and classification techniques may enable the machines to communicate onhigher levels. Machine-machine trust models, collective learning, andknowledge representation are essential for this purpose. Furthermoresome industrial processes may be redefined to optimize the overallperformance in cognitive terms. Both data processing and hardwareconfiguration should result in a secure, reliable, and powerfulprocedure to share information and transfer skills.

Using self-optimizing algorithms for control or parameterization ofindustrial applications offers the possibility to continuously improvethe individual knowledge base. Reinforcement learning, for instance,gives a set of methods that provide this possibility. These algorithmsrely on exploration in the processes state-space in order to learn theoptimal state-action combinations. A reinforcement learning agent canalso be described by a simple P-C-A-Loop, where the process ofevaluating the state information of the environment is the “perception”element of the loop, the alteration of current control laws representsthe “action” part and the process of mapping estimated state informationto new control laws gives the “cognition” section of the single P-C-Aloop. In industrial applications exploring a large state-space is notalways feasible for various reasons like safety, speed, or costs. Usingthe Multi-P-C-A-Loop approach for distributing the learning task overmultiple agents, can reduce the amount of exploration for the individualagents, while the amount of learning experience still remains high. Itfurthermore enables teaching among different P-C-A loops. A possibleassignment for the Multi-P-C-A approach is the combination of multipleagents in one system or assembly line, for instance a monitoring and aclosed-loop control unit. Two different agents could be trained foroptimization of different process parameters. The combination of both ona Multi-P-C-A level could be used to find an optimal path for allparameters.

Both outlined Multi-P-C-A-Loops may improve manufacturing performance insetup and configuration times, process flexibility as well as quality.One approach combines and jointly improves similar workstations withjoint knowledge and skill transfer. The other enables different units toself-improve with each others feedback. In the following, a networkingsystem for cognitive processing devices according to the presentinvention should be described. It is an advantage of the presentinvention, that, once the collaborative systems gain enough machineknowledge, they avoid repetitive configuration steps and maysignificantly reduce down times as well as increase product flexibility.

According to one embodiment of the present invention, in order tofacilitate the integration of several heat treatment monitoring systems100, all distributed systems are connected to each other via internet.The knowledge gained by these systems is shared, thus allowing a globaldatabase of process configurations, sensor setups and qualitybenchmarks.

In order to share information between machines, all of them have to usea similar method of feature acquisition. As a first scenario to achievethese goals using cognitive data processing approaches for combining theinput data from multiple sensors of the respective sensor units 1810 ofthe monitoring systems 100 in order to receive a good estimation of thestate the process is currently in.

Using cognitive dimensionality reduction techniques, unnecessary andredundant data from these sensors can be removed. The reduced sensordata is used to classify the state of the process. Clustering allows foridentification of specific process states, even between differentset-ups. If a significant difference from the references, and thereforean unknown process condition, is detected, the supervisor will bealerted. The expert can then teach the new state and countermeasures (ifpossible) to the system in order to improve its performance.

The cognitive system to be developed should be able to learn to separateacceptable and unacceptable results and furthermore be able to avoidunacceptable results where possible. The usage of technical cognitioneliminates the need for a complete physical model of the baking or foodproduction process. The system is able to stabilize the process byimproving at least one steering variable. Distributed cognition allowsfor a central database between different manufacturing locations. Theinformation gathered from one process can be transferred to a similarprocess at a different location.

1. A heat treatment monitoring system (100), comprising: a heattreatment machine (110) comprising a heat treatment chamber (120), adouble glass window (130) comprising an inside window (136) and anoutside window (135), and an illumination apparatus (140) forilluminating the inside of the heat treatment chamber (120), and amonitoring apparatus (150) mounted to the heat treatment machine (110)and comprising a camera (160) to observe the inside of the heattreatment chamber (120) through the inside window (136), wherein thevisible transmittance of the outside window (135) is lower than thevisible transmittance of the inside window (136) to reduce reflectionswithin the double glass window structure and outside illuminationeffects on image processing of images recorded by the camera (160). 2.The heat treatment monitoring system (100) of claim 1, wherein theoutside window (135) is darkened by a coating.
 3. The heat treatmentmonitoring system (100) of claim 1 or 2, wherein a metal foil or atinting foil is applied at the outside window (135).
 4. The heattreatment monitoring system (100) of any one of the preceding claims,wherein the outside window (135) comprises a tinted glass.
 5. The heattreatment monitoring system (100) of any one of the preceding claims,wherein the outside window (135) has a maximum visible transmittance of60%
 6. The heat treatment monitoring system (100) of any one of thepreceding claims, wherein the double glass window (130) is a heattreatment machine door window (330) of a heat treatment machine door ofthe heat treatment machine (110).
 7. The heat treatment monitoringsystem (100) of any one of the preceding claims, wherein the monitoringapparatus (150) is adapted to generate high dynamic range (HDR)processed images of the food to be heated within the heat treatmentchamber (120).
 8. The heat treatment monitoring system (100) of any oneof the preceding claims, wherein the monitoring apparatus (150) furthercomprises a casing and a camera sensor mount, to which the camera (160)is mounted.
 9. The heat treatment monitoring system (100) of claim 8,wherein the casing is equipped with heat sinks and fans to providecooling of the camera (160).
 10. The heat treatment monitoring system(100) of any one of the preceding claims, wherein the heat treatmentmachine is a convection or a deck oven (310) having at least two traysarranged in a stacked manner.
 11. The heat treatment monitoring system(100) of claim 10, wherein the camera is tilted in such a way in ahorizontal and/or a vertical direction with regard to the double glasswindow (130) to be adapted to observe at least two trays at once in theconvection or deck oven (310).
 12. The heat treatment monitoring system(100) of claim 10, comprising at least two cameras to observe each trayseparately.
 13. The heat treatment monitoring system (100) of any one ofthe preceding claims, further comprising a control unit (1860) beingadapted to process and classify the images of food observed by thecamera based on training data for determining an end time of a heatingprocess for the food.
 14. The heat treatment monitoring system (100) ofclaim 13, wherein the control unit (1860) is adapted to stop the heatingof the heat treatment machine when the heating process has to be ended.15. The heat treatment monitoring system (100) of claim 13, wherein thecontrol unit (1860) is adapted to open automatically the heat treatmentmachine door when the baking process has to be ended, or wherein thecontrol unit is adapted to ventilate the heat treatment chamber withcool air or air when the heating process has to be ended.